-- Logs begin at Thu 2022-09-22 16:26:19 UTC, end at Thu 2022-09-22 17:27:52 UTC. -- Sep 22 16:51:42.072668 dv-cinder-835384-28-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 22 16:51:42.082251 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 22 16:51:42.084326 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 22 16:51:42 2022] *** Sep 22 16:51:42.084326 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 22 16:51:42.084326 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 22 16:51:42.084326 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: nodename: dv-cinder-835384-28-iscsi Sep 22 16:51:42.084326 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: machine: x86_64 Sep 22 16:51:42.084326 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: clock source: unix Sep 22 16:51:42.084326 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: pcre jit disabled Sep 22 16:51:42.084326 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: detected number of CPU cores: 4 Sep 22 16:51:42.084326 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: current working directory: / Sep 22 16:51:42.084326 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: detected binary path: /usr/bin/uwsgi-core Sep 22 16:51:42.084326 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: your processes number limit is 47680 Sep 22 16:51:42.084901 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: your memory page size is 4096 bytes Sep 22 16:51:42.084901 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: detected max file descriptor number: 2048 Sep 22 16:51:42.084901 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: lock engine: pthread robust mutexes Sep 22 16:51:42.084901 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: thunder lock: enabled Sep 22 16:51:42.084901 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 22 16:51:42.084901 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 22 16:51:42.123027 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: Python main interpreter initialized at 0x55bf10fcb9c0 Sep 22 16:51:42.123027 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: python threads support enabled Sep 22 16:51:42.123027 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: your server socket listen backlog is limited to 100 connections Sep 22 16:51:42.123027 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: your mercy for graceful operations on workers is 90 seconds Sep 22 16:51:42.124038 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: mapped 403077 bytes (393 KB) for 2 cores Sep 22 16:51:42.124038 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: *** Operational MODE: preforking *** Sep 22 16:51:42.124220 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: *** uWSGI is running in multiple interpreter mode *** Sep 22 16:51:42.124220 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: spawned uWSGI master process (pid: 81549) Sep 22 16:51:42.124424 dv-cinder-835384-28-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 22 16:51:42.125391 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: spawned uWSGI worker 1 (pid: 81561, cores: 1) Sep 22 16:51:42.125391 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: spawned uWSGI worker 2 (pid: 81562, cores: 1) Sep 22 16:51:42.126568 dv-cinder-835384-28-iscsi devstack@keystone.service[81549]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 22 16:51:46.448414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 16:51:46.448414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: from cryptography.utils import int_from_bytes Sep 22 16:51:46.449213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 16:51:46.449213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: from cryptography.utils import int_from_bytes Sep 22 16:51:46.449672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 16:51:46.449672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: from cryptography.utils import int_from_bytes Sep 22 16:51:46.450123 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 16:51:46.450123 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: from cryptography.utils import int_from_bytes Sep 22 16:51:46.501295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 16:51:46.501502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 16:51:46.501591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 16:51:46.501681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] command line args: [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 16:51:46.501790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 16:51:46.501920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 16:51:46.502006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] command line args: [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 16:51:46.502084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ================================================================================ {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 16:51:46.502251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 16:51:46.502330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] admin_token = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.502436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ================================================================================ {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 16:51:46.502569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] config_dir = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.502697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] admin_token = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.502780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.502925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] config_source = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.503041 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] config_dir = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.503115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] debug = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.503191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.503318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.503617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] config_source = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.503617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] debug = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.503815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.503935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.504062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.504167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.504267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.504376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.504472 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.504574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.504676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.504771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.504878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_config_append = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.504973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.505066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.505168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_config_append = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.505269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_dir = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.505370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.505518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_file = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.505651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_dir = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.505728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.505854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_file = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.505931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.506045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.506126 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.506242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.506316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.506442 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.506518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.506641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.506724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.506841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.506928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.507033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.507152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.507227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.507348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.507435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.507562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.507675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.507793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.507868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.507982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.508056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.508180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.508268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.508352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.508467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.508574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.508653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.508859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.509072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.509256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] publish_errors = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.509338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.509489 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.509609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] publish_errors = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.509719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.509803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.509885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.510000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.510133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.510251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.510384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.510499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.510618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] standard_threads = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.510738 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.510857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.511008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] standard_threads = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.511136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.511375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.511513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.511637 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.511752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] use_journal = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.511865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.512030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] use_json = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.512154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] use_journal = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.512268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] use_stderr = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.512381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] use_json = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.512494 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] use_syslog = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.512608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] use_stderr = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.512733 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.512848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] use_syslog = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.512960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.513073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 16:51:46.513192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.513306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.513431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.513551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.513665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.513779 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.513892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.514005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.514129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.514244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.514357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.514469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.514582 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.514703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.514825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.514939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.515065 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.515177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.515300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.515422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.515547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.515663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.515782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.515896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.516008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.516120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.516291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.516410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.516524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.516637 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.516790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.516902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.517039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.517201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.517345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.517461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.517581 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.517771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.517888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.518001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.518114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.518226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.518338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.518462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.518578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.518696 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.518810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.518930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.519062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.519192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.519307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.519421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.519539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.519652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.519771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.519894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.520008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.520127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.520241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.520360 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.520474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.520594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.520714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.520833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.520948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.521060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.521178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.521334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.521459 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.521581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.521695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.521808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.521921 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.522047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.522168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.522288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.522408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.522522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.522634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.522755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.522869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.523003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.523130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.523244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.523358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.523542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.523664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.523785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.523904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.524019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.524133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.524255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.524369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.524481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.524593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.524712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.524826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.524935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.525031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.525126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.525220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.525321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.525417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.525520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.525633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.525730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.525824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.525938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.526012 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.526135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.526251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.526364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.526516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.526640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.526764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.526884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.527006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.527132 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.527248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.527361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.527482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.527597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.527710 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.527831 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.527952 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.528073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.528196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.528320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.528446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.528562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.528676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.528790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.528903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.529016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.529138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.529260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.529870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.529987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.530108 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.530223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.530353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.530468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.530582 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.530703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.530817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.530930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.531075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.531192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.531305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.531458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.531584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.531700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.531823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.531938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.532052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.532165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.532285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.532401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.532525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.532641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.532755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.532869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.532982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.533095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.533218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.533339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.533454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.533569 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.533693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.533808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.533931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.534047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.534161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.534275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.534388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.534502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.534625 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.534747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.534862 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.535065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.535184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.535297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.535488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.535608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.535722 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.535836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.535949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.536072 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.536196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.536196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.536405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.536519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.536633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.536633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.536847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.536969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.537084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.537198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.537311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.537424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.537537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.537659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.537766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.537883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.537981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.538094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.538216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.538342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.538458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.538571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.538684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.538684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.external = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.538928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.539074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.539192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.539192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.external = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.539429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.539545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.539666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.539666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.password = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.539883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.540006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.token = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.540122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.540236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.540351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.password = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.540465 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.540578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.token = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.540701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.540816 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.540937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.541052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.541167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.541282 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.541388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.541543 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.541661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.541777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.541900 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.542015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.542138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.542255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.542368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.542483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.542598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.542713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.542836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.542967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.543085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.543200 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.543313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.543427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.543554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.543670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.543785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.543899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.544014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.544129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.544251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.544373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.544373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.544590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.544704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.544819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.544819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.545641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.545783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.545941 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.546067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.546185 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.546309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.546426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.546541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.546655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.546769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.546884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.547074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.547206 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.547324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.547702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.547825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.547950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.548075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.548201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.548315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.548430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.548545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.548667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.548783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.548899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.549014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.549129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.549244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.549439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.549562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.549677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.549792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.549915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.550030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.550220 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.550340 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.550455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.550569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.550684 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.550798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.550921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.551057 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.551171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.551285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.551400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.551517 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.551682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.551800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.551915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.552029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.552143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.552264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.552389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.552506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.552621 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.552736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.552850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.552964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.553086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.553202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.553317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.553445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.553561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.553675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.553799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.553915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.554030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.554144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.554258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.554387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.554517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.554634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.554748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.554862 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.554996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.555113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.555241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.555359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.555479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.555597 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.555711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.555825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.555947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.556063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.556176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.556290 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.556404 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.556518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.556641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.556758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.556871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.556984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.557098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.557221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.557337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.557452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.557566 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.557681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.557795 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.557917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.558039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.558155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.558269 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.558383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.558497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.558619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.558737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.558852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.558983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.559101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.559215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.559341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.559459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.559573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.559687 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.559802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.559916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.560039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.560155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.560269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.560383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.560498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.560635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.561248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.561365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.561483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.561598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.561765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.561765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.562000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.562116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.562230 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.562344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.562457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.562457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.562678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.562793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.562908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.563037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.563152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.563152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.563390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.563511 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.563624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.563744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.563859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.563974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.564120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.564244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.564358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.564471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.564584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.564698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.564821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.564937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.565050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.565164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.565277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.565390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.565511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.565627 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.565740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.565853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.565967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.566081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.566203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.566318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.566431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.566545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.566658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.566771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.566894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.567073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.567189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.567303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.567416 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.567529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.567996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.568117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.568232 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.568346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.568460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.568575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.568697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.568815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.568929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.569043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.569159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.569274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.569397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.569513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.569627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.569742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.569857 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.569971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.570094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.570213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.570328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.570442 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.570556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.570671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.570794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.570910 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.571039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.571154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.571270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.571385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.571515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.571633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.571780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.571898 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.572013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.572128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.572252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.572370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.572485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.572600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.572720 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.572845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.572961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.573076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.573190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.573304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.573420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.573609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] role.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.573728 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.573843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] role.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.573957 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.574073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.574186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.574309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.574426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] role.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.574541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.574655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] role.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.574770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.574884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.575062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.575181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.575295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.575408 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.575523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.575636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.575759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.575875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.575989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.575989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.576221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.576847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.576976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.577093 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.577208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.577322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.577437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.577553 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.577676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.577676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.577892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.578007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.578122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.578235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.578359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.578475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.578591 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.578591 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.578806 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.578920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.579063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.579180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.579294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.579409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.579522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.579522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.579750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.579750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.579970 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.580086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.580201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.580324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.580440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.580555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.580670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.580785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.580899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.581021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.581137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.581252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.581366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.581484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.581599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.581723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.581871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.581990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.582112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.582228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.582342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.582466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.582583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.582698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.582814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.582930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.583066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.583195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.583312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.583427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.583542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.583657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.583772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.583896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.584012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.584127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.584242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.584356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.584471 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.584594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.584710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.584824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.584940 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.585055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.585169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.585292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.585408 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.585523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.585638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.585752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.585875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.586001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.586125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.586241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.586355 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.586470 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.connection = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.586584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.586707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.586832 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.586978 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.587109 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.587226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.587341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.587456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.587571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.connection = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.587686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.587810 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.587926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.588042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.588157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.588272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.588387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.588511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.588627 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.588742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.588857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.588971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.589087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.589214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.589332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.589446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.589561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.589675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.589790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.589913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 16:51:46.590996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 16:51:46.590996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 16:51:46.616757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.616989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.617235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.617755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.617994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.618227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.626056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.627144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.654588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.654798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.655658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.655894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.711126 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.711300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.760774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.761087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.761242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.761382 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.918139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.918372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.918489 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.918577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.918732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.918815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.918940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.919047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.919186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.919284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.919407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.919626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.943790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.944017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.944122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.944222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.944300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.944409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.944491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.944639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.944729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.944864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.944945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.945084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.945163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.945312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.956547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.956761 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.969557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.969706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.969844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.969946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.970023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.970151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.970234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.970373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.970464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.970592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.970673 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.970809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.970892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.971067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 16:51:46.988482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:46.988649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:46.988791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 16:51:46.988888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 16:51:46.989029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:46.989176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:46.989258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 16:51:46.989379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 16:51:46.989496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:46.989646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:46.989761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:46.989912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:46.990024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:46.990172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:46.990287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:46.990433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.000496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.000622 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.000786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.000869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.001064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.001221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.001390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.001474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.008392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.008498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.011301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.011399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.011588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.011669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.011873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.011954 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.012155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.012237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.012443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.012523 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.012707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.012795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.012977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.013056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.013251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.013327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.013522 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.013602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.014329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.014415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.014606 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.014683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.037635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.037754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.039125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.039211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.039417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.039519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.044085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.044174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.044373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.044455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.044659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.044740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.051256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.051358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.051548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.051631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.057347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.057457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.057647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.057730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.057943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.058053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.058234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.058324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.058512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.058593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.058792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.058885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.059083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.059166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.059355 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.059435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.074692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.074796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.074998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.075088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.079318 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.079429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.082486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.082611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.082784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.082911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.086933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.087208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.090017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.090414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.093272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.093577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.093826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.093907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.094129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.094214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.094412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.094495 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.094695 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.094775 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.095002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.095295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.110829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.111150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.111420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.111508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.111689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.111785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.112122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.112422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.118551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.120071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.120227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.121628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.121724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.121915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.123252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.123543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.124642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.124936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.125232 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.125551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.125908 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.126260 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.126548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.126840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.127153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.127542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.137054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.137343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.137630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.137917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.138308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.138608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.138698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.139440 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.139568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.139725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.140008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.140391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.140681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.140961 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.154976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.156289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.157911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.159207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.160821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.162091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.162274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.163644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.163767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.164056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.164479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.165121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.165395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.165804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.171562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.172925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.173286 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.174388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.176222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.176624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.176910 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.177194 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.177275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.177669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.177960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.178239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.185056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.185360 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.185694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.186120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.186428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.186719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.192602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.192950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.193184 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.193464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.193585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 16:51:47.193741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.193861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.194018 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.194147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.194295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.194427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.194585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.194717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.194858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.195013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.195158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.195294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.195567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.195663 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.195842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.195923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.196112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.196204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.196388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.196475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.196676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.196970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.197254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 16:51:47.231570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bf10fcb9c0 pid: 81562 (default app) Sep 22 16:51:47.232137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bf10fcb9c0 pid: 81561 (default app) Sep 22 16:51:47.235097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d6932-0459-450f-9fe8-bd77cb3aefa9 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:51:47.235392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d6932-0459-450f-9fe8-bd77cb3aefa9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:51:47.235666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d6932-0459-450f-9fe8-bd77cb3aefa9 None None] PATH_INFO: `/v3` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:51:47.239798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Thu Sep 22 16:51:47 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:51:54.222270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34c109-759c-47a5-a50e-d039e0466cf1 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:51:54.222845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34c109-759c-47a5-a50e-d039e0466cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:51:54.222845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34c109-759c-47a5-a50e-d039e0466cf1 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:51:54.227030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:51:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:51:54.234043 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514cee8-df23-4c2c-9490-013577fe0c34 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:51:54.234671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514cee8-df23-4c2c-9490-013577fe0c34 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:51:54.234671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514cee8-df23-4c2c-9490-013577fe0c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:51:54.305757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: INFO dbcounter [None req-9514cee8-df23-4c2c-9490-013577fe0c34 None None] Registered counter for database keystone Sep 22 16:51:54.325425 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG oslo_db.sqlalchemy.engines [None req-9514cee8-df23-4c2c-9490-013577fe0c34 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81562) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 16:51:54.332573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writer thread running {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 16:51:54.516136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG passlib.handlers.bcrypt [None req-9514cee8-df23-4c2c-9490-013577fe0c34 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81562) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 16:51:54.516451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG passlib.handlers.bcrypt [None req-9514cee8-df23-4c2c-9490-013577fe0c34 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81562) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 16:51:54.574826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9514cee8-df23-4c2c-9490-013577fe0c34 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:51:54.576513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9514cee8-df23-4c2c-9490-013577fe0c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:51:54.704968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:51:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 22 16:51:55.004875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34c109-759c-47a5-a50e-d039e0466cf1 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:51:55.074377 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: INFO dbcounter [None req-ec34c109-759c-47a5-a50e-d039e0466cf1 None None] Registered counter for database keystone Sep 22 16:51:55.096327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG oslo_db.sqlalchemy.engines [None req-ec34c109-759c-47a5-a50e-d039e0466cf1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81561) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 16:51:55.102566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writer thread running {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 16:51:55.222759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34c109-759c-47a5-a50e-d039e0466cf1 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:51:55.227195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b2f9e-a076-4470-b076-a873d7bf90c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:51:55.227822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b2f9e-a076-4470-b076-a873d7bf90c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:51:55.228104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b2f9e-a076-4470-b076-a873d7bf90c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:51:55.228378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b2f9e-a076-4470-b076-a873d7bf90c4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:51:55.237484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b2f9e-a076-4470-b076-a873d7bf90c4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:51:55.290156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a80b2f9e-a076-4470-b076-a873d7bf90c4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 16:51:55.290471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b2f9e-a076-4470-b076-a873d7bf90c4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:51:55.300909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Sep 22 16:51:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 16:51:55.306917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514cee8-df23-4c2c-9490-013577fe0c34 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:51:55.323788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514cee8-df23-4c2c-9490-013577fe0c34 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:51:55.328680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bc17f-5eb4-4292-a02f-6d9706745e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:51:55.329819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bc17f-5eb4-4292-a02f-6d9706745e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:51:55.330102 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bc17f-5eb4-4292-a02f-6d9706745e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:51:55.330375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bc17f-5eb4-4292-a02f-6d9706745e64 admin admin] PATH_INFO: `/v3/projects/5aa999029c6348189112ae4315aae1bd` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:51:55.331991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bc17f-5eb4-4292-a02f-6d9706745e64 admin admin] RBAC: Authorizing `identity:get_project(project_id=5aa999029c6348189112ae4315aae1bd)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:51:55.385307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dc1bc17f-5eb4-4292-a02f-6d9706745e64 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 16:51:55.385656 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bc17f-5eb4-4292-a02f-6d9706745e64 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:51:55.390774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Thu Sep 22 16:51:55 2022] GET /identity/v3/projects/5aa999029c6348189112ae4315aae1bd => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:51:57.541239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb216dfd-194a-47ac-84b4-887270a7f7dd None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:51:57.541819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb216dfd-194a-47ac-84b4-887270a7f7dd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:51:57.541819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb216dfd-194a-47ac-84b4-887270a7f7dd None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:51:57.546378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:51:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:51:57.552612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff41b6-ed00-469c-8628-d342c17cf05e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:51:57.553069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff41b6-ed00-469c-8628-d342c17cf05e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:51:57.553161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff41b6-ed00-469c-8628-d342c17cf05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:51:57.573556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-24ff41b6-ed00-469c-8628-d342c17cf05e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:51:57.575119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-24ff41b6-ed00-469c-8628-d342c17cf05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:51:57.583100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:51:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 22 16:51:57.847382 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb216dfd-194a-47ac-84b4-887270a7f7dd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:51:57.863816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb216dfd-194a-47ac-84b4-887270a7f7dd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:51:57.868368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973947b3-a31b-4ea5-968b-4a401afc8a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:51:57.868976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973947b3-a31b-4ea5-968b-4a401afc8a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:51:57.869254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973947b3-a31b-4ea5-968b-4a401afc8a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:51:57.869522 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973947b3-a31b-4ea5-968b-4a401afc8a1d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:51:57.878163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973947b3-a31b-4ea5-968b-4a401afc8a1d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:51:57.879954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973947b3-a31b-4ea5-968b-4a401afc8a1d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:51:57.888692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Sep 22 16:51:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 16:51:57.895015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff41b6-ed00-469c-8628-d342c17cf05e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:51:57.904186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff41b6-ed00-469c-8628-d342c17cf05e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:51:57.908544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0c58e-d031-456d-9d42-89f58a48464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:51:57.909175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0c58e-d031-456d-9d42-89f58a48464a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:51:57.909456 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0c58e-d031-456d-9d42-89f58a48464a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:51:57.909726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0c58e-d031-456d-9d42-89f58a48464a admin admin] PATH_INFO: `/v3/users/d723a6acb5a24745b0576ef9d3892945` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:51:57.911080 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0c58e-d031-456d-9d42-89f58a48464a admin admin] RBAC: Authorizing `identity:get_user(user_id=d723a6acb5a24745b0576ef9d3892945)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:51:57.912934 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0c58e-d031-456d-9d42-89f58a48464a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:51:57.914478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Thu Sep 22 16:51:57 2022] GET /identity/v3/users/d723a6acb5a24745b0576ef9d3892945 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:00.142112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a829f2-533b-4feb-ab19-8b105d518984 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:00.142619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a829f2-533b-4feb-ab19-8b105d518984 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:00.142619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a829f2-533b-4feb-ab19-8b105d518984 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:00.143980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:00.149490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9d961-09a2-44a6-8c4e-e043d78518d2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:00.149921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9d961-09a2-44a6-8c4e-e043d78518d2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:00.150027 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9d961-09a2-44a6-8c4e-e043d78518d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:00.168690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-32a9d961-09a2-44a6-8c4e-e043d78518d2 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:00.170497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-32a9d961-09a2-44a6-8c4e-e043d78518d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:00.196809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e9ed1-035a-4b26-a6cf-757eee648c3d None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:00.197087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e9ed1-035a-4b26-a6cf-757eee648c3d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:00.197362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e9ed1-035a-4b26-a6cf-757eee648c3d None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:00.198674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:00.204271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d409c2b-66dc-45d8-aa9f-0b350b161a05 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:00.204554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d409c2b-66dc-45d8-aa9f-0b350b161a05 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:00.204828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d409c2b-66dc-45d8-aa9f-0b350b161a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:00.248481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG passlib.handlers.bcrypt [None req-6d409c2b-66dc-45d8-aa9f-0b350b161a05 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81561) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 16:52:00.248881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG passlib.handlers.bcrypt [None req-6d409c2b-66dc-45d8-aa9f-0b350b161a05 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81561) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 16:52:00.255902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:00.298833 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6d409c2b-66dc-45d8-aa9f-0b350b161a05 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:00.300524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6d409c2b-66dc-45d8-aa9f-0b350b161a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:00.401358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:00.525443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9d961-09a2-44a6-8c4e-e043d78518d2 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:00.550150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:00.632485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9d961-09a2-44a6-8c4e-e043d78518d2 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:00.666153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80b1f2-d512-4260-8be9-ed3a2af7d13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:00.666707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80b1f2-d512-4260-8be9-ed3a2af7d13c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:00.667010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80b1f2-d512-4260-8be9-ed3a2af7d13c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:00.667287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80b1f2-d512-4260-8be9-ed3a2af7d13c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:00.669479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d409c2b-66dc-45d8-aa9f-0b350b161a05 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:00.680928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80b1f2-d512-4260-8be9-ed3a2af7d13c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-7d80b1f2-d512-4260-8be9-ed3a2af7d13c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:00.689773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:00.690864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:00.691971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:52:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:00.698664 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80b1f2-d512-4260-8be9-ed3a2af7d13c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:00.778273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d409c2b-66dc-45d8-aa9f-0b350b161a05 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:00.780618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:00.786407 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:00.806225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80b1f2-d512-4260-8be9-ed3a2af7d13c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:00.823714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09cdb-3b5d-4b58-bb57-12cb326c1132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:00.824599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09cdb-3b5d-4b58-bb57-12cb326c1132 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:00.824888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09cdb-3b5d-4b58-bb57-12cb326c1132 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:00.825161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09cdb-3b5d-4b58-bb57-12cb326c1132 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:00.842689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9dd46-d4c6-4c2d-a745-8497eaa1d3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:00.843232 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9dd46-d4c6-4c2d-a745-8497eaa1d3da None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:00.843519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9dd46-d4c6-4c2d-a745-8497eaa1d3da None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:00.843798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9dd46-d4c6-4c2d-a745-8497eaa1d3da None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:00.844506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9dd46-d4c6-4c2d-a745-8497eaa1d3da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:00.844814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9dd46-d4c6-4c2d-a745-8497eaa1d3da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:00.846288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9dd46-d4c6-4c2d-a745-8497eaa1d3da None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:00.859767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:00.865786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9dd46-d4c6-4c2d-a745-8497eaa1d3da None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:00.902018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d09cdb-3b5d-4b58-bb57-12cb326c1132 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:00.903603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d09cdb-3b5d-4b58-bb57-12cb326c1132 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:00.973340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9dd46-d4c6-4c2d-a745-8497eaa1d3da None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:01.009093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069867b9-344d-43e9-8c42-7737fef56918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:01.009739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069867b9-344d-43e9-8c42-7737fef56918 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:01.009893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 16:52:01.010065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069867b9-344d-43e9-8c42-7737fef56918 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:01.010294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069867b9-344d-43e9-8c42-7737fef56918 None admin] PATH_INFO: `/v3/users/d723a6acb5a24745b0576ef9d3892945` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:01.011485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069867b9-344d-43e9-8c42-7737fef56918 None admin] RBAC: Authorizing `identity:get_user(user_id=d723a6acb5a24745b0576ef9d3892945)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:01.012955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069867b9-344d-43e9-8c42-7737fef56918 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:01.015120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:00 2022] GET /identity/v3/users/d723a6acb5a24745b0576ef9d3892945 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:01.016178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09cdb-3b5d-4b58-bb57-12cb326c1132 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:01.019147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:01.020345 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069867b9-344d-43e9-8c42-7737fef56918 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:01.035298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:01.118145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09cdb-3b5d-4b58-bb57-12cb326c1132 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:01.125698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069867b9-344d-43e9-8c42-7737fef56918 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:01.155120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b881b-ec8f-4285-be8a-99f331c37424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:01.156142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b881b-ec8f-4285-be8a-99f331c37424 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:01.156738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b881b-ec8f-4285-be8a-99f331c37424 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:01.157016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b881b-ec8f-4285-be8a-99f331c37424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:01.163889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c588a3-e3bd-4ec6-af00-7cdc5fefe48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:01.164522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c588a3-e3bd-4ec6-af00-7cdc5fefe48b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:01.164795 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c588a3-e3bd-4ec6-af00-7cdc5fefe48b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:01.165060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c588a3-e3bd-4ec6-af00-7cdc5fefe48b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:01.166173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c588a3-e3bd-4ec6-af00-7cdc5fefe48b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:01.167637 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c588a3-e3bd-4ec6-af00-7cdc5fefe48b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:01.169464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 10/18] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:01.175700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c588a3-e3bd-4ec6-af00-7cdc5fefe48b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:01.176377 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b881b-ec8f-4285-be8a-99f331c37424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:01.177855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b881b-ec8f-4285-be8a-99f331c37424 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b23b881b-ec8f-4285-be8a-99f331c37424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:52:01.197074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:01.198151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:52:01.199039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:52:01.199039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:52:01.199039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:01.199039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 9/19] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:52:01.204297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b881b-ec8f-4285-be8a-99f331c37424 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:01.260956 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:01.262041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:01.280916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c588a3-e3bd-4ec6-af00-7cdc5fefe48b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:01.308275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b881b-ec8f-4285-be8a-99f331c37424 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:01.316116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a64c46-cb49-42fc-9f4a-85ececac716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:01.316719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a64c46-cb49-42fc-9f4a-85ececac716c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:01.317002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a64c46-cb49-42fc-9f4a-85ececac716c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:01.317277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a64c46-cb49-42fc-9f4a-85ececac716c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:01.318003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a64c46-cb49-42fc-9f4a-85ececac716c None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.domain.id=default, user.id=d723a6acb5a24745b0576ef9d3892945) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:01.318309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a64c46-cb49-42fc-9f4a-85ececac716c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:01.319765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a64c46-cb49-42fc-9f4a-85ececac716c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:01.326909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Sep 22 16:52:01 2022] GET /identity/v3/role_assignments?user.id=d723a6acb5a24745b0576ef9d3892945&scope.domain.id=default&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:01.344185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9c58a-bb8c-4b50-bdca-166a4a2f1652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:01.344821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9c58a-bb8c-4b50-bdca-166a4a2f1652 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:01.345099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9c58a-bb8c-4b50-bdca-166a4a2f1652 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:01.345371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9c58a-bb8c-4b50-bdca-166a4a2f1652 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:01.346034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9c58a-bb8c-4b50-bdca-166a4a2f1652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:01.346336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9c58a-bb8c-4b50-bdca-166a4a2f1652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:01.347794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9c58a-bb8c-4b50-bdca-166a4a2f1652 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:01.367407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:03.774649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5521b10-5332-4a47-a06a-3fd7debad634 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:03.775141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5521b10-5332-4a47-a06a-3fd7debad634 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:03.775251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5521b10-5332-4a47-a06a-3fd7debad634 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:03.776675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:03.783045 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2e9e3-7630-46b1-8f58-df5d02cb4a3a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:03.783928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2e9e3-7630-46b1-8f58-df5d02cb4a3a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:03.784393 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2e9e3-7630-46b1-8f58-df5d02cb4a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:03.793623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f941ef-ac6a-40fb-a2f6-efccc873e692 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:03.793902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f941ef-ac6a-40fb-a2f6-efccc873e692 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:03.794401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f941ef-ac6a-40fb-a2f6-efccc873e692 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:03.796548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 13/23] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:03.802346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55bf51-d489-4846-ba9a-72667bc8dbbe None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:03.802912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55bf51-d489-4846-ba9a-72667bc8dbbe None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:03.803413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55bf51-d489-4846-ba9a-72667bc8dbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:03.813110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-dec2e9e3-7630-46b1-8f58-df5d02cb4a3a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:03.814325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dec2e9e3-7630-46b1-8f58-df5d02cb4a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:03.828793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0d55bf51-d489-4846-ba9a-72667bc8dbbe None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:03.830310 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0d55bf51-d489-4846-ba9a-72667bc8dbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:03.973280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 11/24] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:03.981250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 14/25] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:04.309693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2e9e3-7630-46b1-8f58-df5d02cb4a3a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:04.367531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:04.435207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2e9e3-7630-46b1-8f58-df5d02cb4a3a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:04.477132 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfcb9f8-3f3d-496a-9d08-f805801a930e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:04.477790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcb9f8-3f3d-496a-9d08-f805801a930e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:04.478072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcb9f8-3f3d-496a-9d08-f805801a930e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:04.478343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcb9f8-3f3d-496a-9d08-f805801a930e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:04.490268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfcb9f8-3f3d-496a-9d08-f805801a930e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:04.491783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfcb9f8-3f3d-496a-9d08-f805801a930e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-0cfcb9f8-3f3d-496a-9d08-f805801a930e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:52:04.513422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:04.514531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:52:04.515400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:52:04.515400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:52:04.515400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:04.515400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 12/26] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:04.524068 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55bf51-d489-4846-ba9a-72667bc8dbbe None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:04.595200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfcb9f8-3f3d-496a-9d08-f805801a930e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:04.639311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:04.682783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55bf51-d489-4846-ba9a-72667bc8dbbe None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:04.783315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:04.784610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11507b5f-d602-40a1-b0ca-6bbd9dd92483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:04.785262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11507b5f-d602-40a1-b0ca-6bbd9dd92483 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:04.785610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11507b5f-d602-40a1-b0ca-6bbd9dd92483 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:04.785810 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11507b5f-d602-40a1-b0ca-6bbd9dd92483 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:04.786617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11507b5f-d602-40a1-b0ca-6bbd9dd92483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:04.786917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11507b5f-d602-40a1-b0ca-6bbd9dd92483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:04.788397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11507b5f-d602-40a1-b0ca-6bbd9dd92483 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:04.788956 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfcb9f8-3f3d-496a-9d08-f805801a930e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:04.821527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 15/27] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:04.826976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11507b5f-d602-40a1-b0ca-6bbd9dd92483 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:04.865269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf54e-673e-4a3b-81d0-cdc3e37484f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:04.865842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf54e-673e-4a3b-81d0-cdc3e37484f0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:04.866122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf54e-673e-4a3b-81d0-cdc3e37484f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:04.866389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf54e-673e-4a3b-81d0-cdc3e37484f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:04.896126 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaf54e-673e-4a3b-81d0-cdc3e37484f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-3dcaf54e-673e-4a3b-81d0-cdc3e37484f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:04.903086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:04.903951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:04.904811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 13/28] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:52:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:52:04.912447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf54e-673e-4a3b-81d0-cdc3e37484f0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:05.020468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11507b5f-d602-40a1-b0ca-6bbd9dd92483 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:05.058215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6895d-9dae-467f-aa2c-900b7a493e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:05.058929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6895d-9dae-467f-aa2c-900b7a493e86 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.059241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6895d-9dae-467f-aa2c-900b7a493e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.059519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6895d-9dae-467f-aa2c-900b7a493e86 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.060180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6895d-9dae-467f-aa2c-900b7a493e86 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:05.061820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6895d-9dae-467f-aa2c-900b7a493e86 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:05.067239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:05.076758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf54e-673e-4a3b-81d0-cdc3e37484f0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:05.117742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dd357-0512-4cbc-8b5d-20bece8c0a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:05.118275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dd357-0512-4cbc-8b5d-20bece8c0a78 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.118554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dd357-0512-4cbc-8b5d-20bece8c0a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.118823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dd357-0512-4cbc-8b5d-20bece8c0a78 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.119527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dd357-0512-4cbc-8b5d-20bece8c0a78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:05.119825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dd357-0512-4cbc-8b5d-20bece8c0a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:05.121206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dd357-0512-4cbc-8b5d-20bece8c0a78 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:05.141882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 14/29] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:05.143397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8de20-2f0f-4532-9180-7994319e60a5 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.143682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8de20-2f0f-4532-9180-7994319e60a5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.143956 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8de20-2f0f-4532-9180-7994319e60a5 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.145239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 15/30] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:05.146291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5613c5-6671-4e96-9840-f26b6afbada7 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.146569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5613c5-6671-4e96-9840-f26b6afbada7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.146834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5613c5-6671-4e96-9840-f26b6afbada7 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.148004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 16/31] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:05.148691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5613c5-6671-4e96-9840-f26b6afbada7 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:05.173733 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:05.220338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 16/32] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:52:04 2022] POST /identity/v3/projects => generated 301 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 16:52:05.222052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd935e-e51e-4607-bd32-066fffaa16b9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.222302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd935e-e51e-4607-bd32-066fffaa16b9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.222585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd935e-e51e-4607-bd32-066fffaa16b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.245434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-09bd935e-e51e-4607-bd32-066fffaa16b9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:05.247225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-09bd935e-e51e-4607-bd32-066fffaa16b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:05.270295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5613c5-6671-4e96-9840-f26b6afbada7 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:05.312995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426ff2d-fcff-4c22-8ae0-76793ef6175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:05.314086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426ff2d-fcff-4c22-8ae0-76793ef6175b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.314755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426ff2d-fcff-4c22-8ae0-76793ef6175b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.315195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426ff2d-fcff-4c22-8ae0-76793ef6175b None admin] PATH_INFO: `/v3/users/d723a6acb5a24745b0576ef9d3892945` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.317067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426ff2d-fcff-4c22-8ae0-76793ef6175b None admin] RBAC: Authorizing `identity:get_user(user_id=d723a6acb5a24745b0576ef9d3892945)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:05.318971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426ff2d-fcff-4c22-8ae0-76793ef6175b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:05.320830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 17/33] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:05 2022] GET /identity/v3/users/d723a6acb5a24745b0576ef9d3892945 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:05.322103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d2b61-8f92-4866-843b-75ef90aea65a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.322601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d2b61-8f92-4866-843b-75ef90aea65a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.323051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d2b61-8f92-4866-843b-75ef90aea65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.335133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 17/34] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:05.336183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd935e-e51e-4607-bd32-066fffaa16b9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:05.345579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e87d2b61-8f92-4866-843b-75ef90aea65a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:05.347281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e87d2b61-8f92-4866-843b-75ef90aea65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:05.531755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 18/35] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:05.558923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd935e-e51e-4607-bd32-066fffaa16b9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:05.583137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:05.584395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e00bb2-51c4-4840-a389-c61083060cda None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.584688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e00bb2-51c4-4840-a389-c61083060cda None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.584970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e00bb2-51c4-4840-a389-c61083060cda None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.586332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 19/36] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:05.593889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e89e5-1787-4776-a85d-e59de02c37ad None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.594176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e89e5-1787-4776-a85d-e59de02c37ad None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.594443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e89e5-1787-4776-a85d-e59de02c37ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.619198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95795a0a-b0f9-49c2-a3ec-07498c9207d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:05.619858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95795a0a-b0f9-49c2-a3ec-07498c9207d2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.620147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95795a0a-b0f9-49c2-a3ec-07498c9207d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.620424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95795a0a-b0f9-49c2-a3ec-07498c9207d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.625140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95795a0a-b0f9-49c2-a3ec-07498c9207d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:05.626774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95795a0a-b0f9-49c2-a3ec-07498c9207d2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:05.628401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 18/37] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:05.629408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f91e89e5-1787-4776-a85d-e59de02c37ad None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:05.630670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f91e89e5-1787-4776-a85d-e59de02c37ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:05.634186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95795a0a-b0f9-49c2-a3ec-07498c9207d2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:05.657737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:05.771980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 20/38] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:05.772600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e89e5-1787-4776-a85d-e59de02c37ad None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:05.863903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95795a0a-b0f9-49c2-a3ec-07498c9207d2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:05.899034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e89e5-1787-4776-a85d-e59de02c37ad None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:05.909954 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de26dd-0f74-4841-bade-80f01961266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:05.911304 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de26dd-0f74-4841-bade-80f01961266c None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.912037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de26dd-0f74-4841-bade-80f01961266c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.912454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de26dd-0f74-4841-bade-80f01961266c None admin] PATH_INFO: `/v3/domains/default/users/d723a6acb5a24745b0576ef9d3892945/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.915307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de26dd-0f74-4841-bade-80f01961266c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d723a6acb5a24745b0576ef9d3892945, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:05.917252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de26dd-0f74-4841-bade-80f01961266c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:05.937372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 19/39] 10.222.0.16 () {58 vars in 1361 bytes} [Thu Sep 22 16:52:05 2022] PUT /identity/v3/domains/default/users/d723a6acb5a24745b0576ef9d3892945/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 16:52:05.938435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb169c-bc4e-4efd-acbd-cdb90adcbeb5 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.938729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb169c-bc4e-4efd-acbd-cdb90adcbeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.939052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb169c-bc4e-4efd-acbd-cdb90adcbeb5 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.944573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 20/40] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:05.944573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb169c-bc4e-4efd-acbd-cdb90adcbeb5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:05.947752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30222e8-18af-441d-aa74-043efc37a725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:05.948997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30222e8-18af-441d-aa74-043efc37a725 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:05.950780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30222e8-18af-441d-aa74-043efc37a725 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:05.951345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30222e8-18af-441d-aa74-043efc37a725 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:05.952234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30222e8-18af-441d-aa74-043efc37a725 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:05.953847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30222e8-18af-441d-aa74-043efc37a725 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:06.012212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:06.086619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb169c-bc4e-4efd-acbd-cdb90adcbeb5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:06.109988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 21/41] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 22 16:52:05 2022] POST /identity/v3/roles => generated 225 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:06.111817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2482-1c55-4c55-96e0-5f877dc941ca None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:06.112321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2482-1c55-4c55-96e0-5f877dc941ca None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:06.112739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2482-1c55-4c55-96e0-5f877dc941ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:06.141517 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a09a2482-1c55-4c55-96e0-5f877dc941ca None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:06.142223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00630df5-8b58-4d9d-83cc-e5ffecf69c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:06.143129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a09a2482-1c55-4c55-96e0-5f877dc941ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:06.143288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00630df5-8b58-4d9d-83cc-e5ffecf69c06 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:06.143697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00630df5-8b58-4d9d-83cc-e5ffecf69c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:06.144111 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00630df5-8b58-4d9d-83cc-e5ffecf69c06 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:06.144985 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00630df5-8b58-4d9d-83cc-e5ffecf69c06 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:06.145735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:06.146445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00630df5-8b58-4d9d-83cc-e5ffecf69c06 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:06.167817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 21/42] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 22 16:52:05 2022] POST /identity/v3/roles => generated 219 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:06.191016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00630df5-8b58-4d9d-83cc-e5ffecf69c06 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:06.237841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 22/43] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:06.459086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00630df5-8b58-4d9d-83cc-e5ffecf69c06 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:06.467593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:06.499081 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a57bd1-38be-4c4f-b8c5-0bc027f56c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:06.499719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a57bd1-38be-4c4f-b8c5-0bc027f56c75 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:06.500002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a57bd1-38be-4c4f-b8c5-0bc027f56c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:06.500301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a57bd1-38be-4c4f-b8c5-0bc027f56c75 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:06.501016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a57bd1-38be-4c4f-b8c5-0bc027f56c75 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:06.502497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a57bd1-38be-4c4f-b8c5-0bc027f56c75 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:06.522082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 22/44] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 22 16:52:06 2022] POST /identity/v3/roles => generated 223 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 16:52:06.610320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a2482-1c55-4c55-96e0-5f877dc941ca None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:06.768121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a2482-1c55-4c55-96e0-5f877dc941ca None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:06.808593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ba648-a023-4869-9095-331948a338dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:06.809696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ba648-a023-4869-9095-331948a338dc None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:06.810121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ba648-a023-4869-9095-331948a338dc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:06.810560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ba648-a023-4869-9095-331948a338dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:06.811776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ba648-a023-4869-9095-331948a338dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:06.813404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ba648-a023-4869-9095-331948a338dc None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:06.816486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 23/45] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:06.824217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a57bd1-38be-4c4f-b8c5-0bc027f56c75 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:06.928677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a57bd1-38be-4c4f-b8c5-0bc027f56c75 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:06.961782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ab597-9849-4301-adae-8189a9bc9692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:06.962402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab597-9849-4301-adae-8189a9bc9692 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:06.962748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab597-9849-4301-adae-8189a9bc9692 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:06.963063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab597-9849-4301-adae-8189a9bc9692 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:06.963737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ab597-9849-4301-adae-8189a9bc9692 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:06.965209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ab597-9849-4301-adae-8189a9bc9692 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:06.998175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 23/46] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:52:06 2022] POST /identity/v3/projects => generated 312 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:08.316986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf8e6f-cffe-4f3c-a1ab-8c2782a2edb5 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:08.317600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf8e6f-cffe-4f3c-a1ab-8c2782a2edb5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:08.317600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf8e6f-cffe-4f3c-a1ab-8c2782a2edb5 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:08.318892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 24/47] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:08.324767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570827d-850d-4eed-9c07-0b47ea1d29a9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:08.325279 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570827d-850d-4eed-9c07-0b47ea1d29a9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:08.325379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570827d-850d-4eed-9c07-0b47ea1d29a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:08.346128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2570827d-850d-4eed-9c07-0b47ea1d29a9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:08.347281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2570827d-850d-4eed-9c07-0b47ea1d29a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:08.359496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:08.418126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 24/48] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:08.691138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf8e6f-cffe-4f3c-a1ab-8c2782a2edb5 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:08.700100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:08.798571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf8e6f-cffe-4f3c-a1ab-8c2782a2edb5 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:08.832548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbe49e-33f0-4802-8637-7461a09d041f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:08.833232 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbe49e-33f0-4802-8637-7461a09d041f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:08.833550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbe49e-33f0-4802-8637-7461a09d041f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:08.833828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbe49e-33f0-4802-8637-7461a09d041f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:08.845200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbe49e-33f0-4802-8637-7461a09d041f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-d2bbe49e-33f0-4802-8637-7461a09d041f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:08.850124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:08.850927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:08.851689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 25/49] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:52:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:08.857256 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570827d-850d-4eed-9c07-0b47ea1d29a9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:08.955515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570827d-850d-4eed-9c07-0b47ea1d29a9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:08.988681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afa8e0-0da4-453d-884a-16bf7643f008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:08.989292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afa8e0-0da4-453d-884a-16bf7643f008 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:08.989575 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afa8e0-0da4-453d-884a-16bf7643f008 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:08.989849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afa8e0-0da4-453d-884a-16bf7643f008 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:08.990513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afa8e0-0da4-453d-884a-16bf7643f008 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:08.990819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afa8e0-0da4-453d-884a-16bf7643f008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:08.992256 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afa8e0-0da4-453d-884a-16bf7643f008 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:09.000768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 25/50] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:09.006494 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbe49e-33f0-4802-8637-7461a09d041f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:09.072310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:09.111512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbe49e-33f0-4802-8637-7461a09d041f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:09.146546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9825f28-3688-412e-b3bb-a33b1008e76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:09.146546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9825f28-3688-412e-b3bb-a33b1008e76f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:09.147544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9825f28-3688-412e-b3bb-a33b1008e76f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:09.147750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9825f28-3688-412e-b3bb-a33b1008e76f None admin] PATH_INFO: `/v3/users/d723a6acb5a24745b0576ef9d3892945` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:09.148959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9825f28-3688-412e-b3bb-a33b1008e76f None admin] RBAC: Authorizing `identity:get_user(user_id=d723a6acb5a24745b0576ef9d3892945)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:09.150407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9825f28-3688-412e-b3bb-a33b1008e76f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:09.152038 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 26/51] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:09 2022] GET /identity/v3/users/d723a6acb5a24745b0576ef9d3892945 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:09.158317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afa8e0-0da4-453d-884a-16bf7643f008 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:09.170377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:09.193592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c0f3-6703-4280-a4ae-311cb49cdbea None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:09.193873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c0f3-6703-4280-a4ae-311cb49cdbea None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:09.194146 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c0f3-6703-4280-a4ae-311cb49cdbea None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:09.195500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 27/52] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:09.201362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95277ca-48bf-4d3f-b3bc-a7b004004efb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:09.201646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95277ca-48bf-4d3f-b3bc-a7b004004efb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:09.201922 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95277ca-48bf-4d3f-b3bc-a7b004004efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:09.220678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e95277ca-48bf-4d3f-b3bc-a7b004004efb None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:09.221832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e95277ca-48bf-4d3f-b3bc-a7b004004efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:09.267451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afa8e0-0da4-453d-884a-16bf7643f008 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:09.292901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 28/53] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:09.302549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef501cd2-b512-4ab1-8bd9-20ac0a7519c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:09.303224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef501cd2-b512-4ab1-8bd9-20ac0a7519c4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:09.303513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef501cd2-b512-4ab1-8bd9-20ac0a7519c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:09.303799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef501cd2-b512-4ab1-8bd9-20ac0a7519c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:09.304920 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef501cd2-b512-4ab1-8bd9-20ac0a7519c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:09.306376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef501cd2-b512-4ab1-8bd9-20ac0a7519c4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:09.307976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 26/54] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:09.314224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95277ca-48bf-4d3f-b3bc-a7b004004efb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:09.483279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:09.523354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95277ca-48bf-4d3f-b3bc-a7b004004efb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:09.557057 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1566de-10ef-40d4-b54d-f2543791d39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:09.557662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1566de-10ef-40d4-b54d-f2543791d39e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:09.557938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1566de-10ef-40d4-b54d-f2543791d39e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:09.558211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1566de-10ef-40d4-b54d-f2543791d39e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:09.558935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1566de-10ef-40d4-b54d-f2543791d39e None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.domain.id=default, user.id=d723a6acb5a24745b0576ef9d3892945) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:09.559268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1566de-10ef-40d4-b54d-f2543791d39e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:09.560746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1566de-10ef-40d4-b54d-f2543791d39e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:09.569232 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 29/55] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Sep 22 16:52:09 2022] GET /identity/v3/role_assignments?user.id=d723a6acb5a24745b0576ef9d3892945&scope.domain.id=default&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 530 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:09.572300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef501cd2-b512-4ab1-8bd9-20ac0a7519c4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:09.661758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:09.687047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef501cd2-b512-4ab1-8bd9-20ac0a7519c4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:09.723448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c11e8-626d-42b6-99e5-212f1823c763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:09.724114 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c11e8-626d-42b6-99e5-212f1823c763 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:09.724402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c11e8-626d-42b6-99e5-212f1823c763 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:09.724678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c11e8-626d-42b6-99e5-212f1823c763 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:09.725835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c11e8-626d-42b6-99e5-212f1823c763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:09.727349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c11e8-626d-42b6-99e5-212f1823c763 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:09.729015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 27/56] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:09.735458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1566de-10ef-40d4-b54d-f2543791d39e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:09.840198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1566de-10ef-40d4-b54d-f2543791d39e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:09.866958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:09.874681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae9ef6-8733-455b-be29-af9239de6bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:09.875315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae9ef6-8733-455b-be29-af9239de6bfd None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:09.875596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae9ef6-8733-455b-be29-af9239de6bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:09.875864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae9ef6-8733-455b-be29-af9239de6bfd None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:09.876502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae9ef6-8733-455b-be29-af9239de6bfd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:09.877930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae9ef6-8733-455b-be29-af9239de6bfd None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:09.905531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 30/57] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Sep 22 16:52:09 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:12.137658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508b634-d8ca-4732-a00f-58b5ed4e91f8 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:12.138152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508b634-d8ca-4732-a00f-58b5ed4e91f8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:12.138152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508b634-d8ca-4732-a00f-58b5ed4e91f8 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:12.139661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 28/58] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:12.146647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56286a1-a7f8-4d36-8c72-8577a68230e3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:12.147206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56286a1-a7f8-4d36-8c72-8577a68230e3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:12.147277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56286a1-a7f8-4d36-8c72-8577a68230e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:12.166929 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b56286a1-a7f8-4d36-8c72-8577a68230e3 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:12.168215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b56286a1-a7f8-4d36-8c72-8577a68230e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:12.248801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 31/59] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:12.526676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508b634-d8ca-4732-a00f-58b5ed4e91f8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:12.638169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508b634-d8ca-4732-a00f-58b5ed4e91f8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:12.672510 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6a562-b02f-4e27-bb49-b2f1fe902be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:12.673407 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6a562-b02f-4e27-bb49-b2f1fe902be5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:12.673687 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6a562-b02f-4e27-bb49-b2f1fe902be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:12.673966 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6a562-b02f-4e27-bb49-b2f1fe902be5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:12.675131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6a562-b02f-4e27-bb49-b2f1fe902be5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:12.676583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6a562-b02f-4e27-bb49-b2f1fe902be5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:12.678176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 29/60] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:12.683955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56286a1-a7f8-4d36-8c72-8577a68230e3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:12.787555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56286a1-a7f8-4d36-8c72-8577a68230e3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:12.799562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:12.825305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2c35f-d2e7-424c-bd77-887f760d0a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:12.826471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c35f-d2e7-424c-bd77-887f760d0a85 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:12.826752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c35f-d2e7-424c-bd77-887f760d0a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:12.827054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c35f-d2e7-424c-bd77-887f760d0a85 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:12.827719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2c35f-d2e7-424c-bd77-887f760d0a85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:12.829180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2c35f-d2e7-424c-bd77-887f760d0a85 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:12.861689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 32/61] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:52:12 2022] POST /identity/v3/users => generated 275 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:16.518697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1909be-24cb-46fc-8575-1f83df73bf6e None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:16.518697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1909be-24cb-46fc-8575-1f83df73bf6e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:16.518697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1909be-24cb-46fc-8575-1f83df73bf6e None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:16.518697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 30/62] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:16.533545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa979d93-06d1-436e-acc5-d4f511d06d78 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:16.534055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa979d93-06d1-436e-acc5-d4f511d06d78 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:16.534370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa979d93-06d1-436e-acc5-d4f511d06d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:16.561433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-fa979d93-06d1-436e-acc5-d4f511d06d78 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:16.562667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fa979d93-06d1-436e-acc5-d4f511d06d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:16.659140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0cc94-1ca8-4dce-9225-f6bbcd2d6c4d None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:16.659140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0cc94-1ca8-4dce-9225-f6bbcd2d6c4d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:16.659140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0cc94-1ca8-4dce-9225-f6bbcd2d6c4d None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:16.659140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 31/63] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:16.661973 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 33/64] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:16.667734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7c96f-c1b3-4aa0-ac72-df6b2b5ed27c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:16.668336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7c96f-c1b3-4aa0-ac72-df6b2b5ed27c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:16.668814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7c96f-c1b3-4aa0-ac72-df6b2b5ed27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:16.696041 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8fe7c96f-c1b3-4aa0-ac72-df6b2b5ed27c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:16.697702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8fe7c96f-c1b3-4aa0-ac72-df6b2b5ed27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:16.711005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:16.716871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cc79e-2306-4ef4-b5a8-dbeb2692cec9 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:16.717369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cc79e-2306-4ef4-b5a8-dbeb2692cec9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:16.717781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cc79e-2306-4ef4-b5a8-dbeb2692cec9 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:16.719475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 34/65] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:16.726401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661244ab-0e14-4d14-bc3c-1b77f4c98887 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:16.726703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661244ab-0e14-4d14-bc3c-1b77f4c98887 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:16.727014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661244ab-0e14-4d14-bc3c-1b77f4c98887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:16.768235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-661244ab-0e14-4d14-bc3c-1b77f4c98887 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:16.769510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-661244ab-0e14-4d14-bc3c-1b77f4c98887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:16.829859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 32/66] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:16.836534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ae60e-a834-4774-b514-dc165e0666ba None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:16.836827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ae60e-a834-4774-b514-dc165e0666ba None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:16.837111 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ae60e-a834-4774-b514-dc165e0666ba None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:16.838440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 33/67] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:16.852973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b510b-929d-49c1-8e75-130ac288cc31 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:16.853253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b510b-929d-49c1-8e75-130ac288cc31 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:16.853780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b510b-929d-49c1-8e75-130ac288cc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:16.884643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5f0b510b-929d-49c1-8e75-130ac288cc31 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:16.887724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5f0b510b-929d-49c1-8e75-130ac288cc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:16.907875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 35/68] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 22 16:52:16.974502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661244ab-0e14-4d14-bc3c-1b77f4c98887 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:16.975266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 34/69] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:17.042882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:17.155635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661244ab-0e14-4d14-bc3c-1b77f4c98887 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:17.217928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c054543-b363-4451-b052-809d55a4117f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:17.218671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c054543-b363-4451-b052-809d55a4117f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:17.218981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c054543-b363-4451-b052-809d55a4117f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:17.219247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c054543-b363-4451-b052-809d55a4117f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:17.235957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c054543-b363-4451-b052-809d55a4117f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:17.240974 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff5d84-a996-4b58-9377-820c5254cba3 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:17.241562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff5d84-a996-4b58-9377-820c5254cba3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-4c054543-b363-4451-b052-809d55a4117f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:17.241842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:17.242687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:17.249948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 36/70] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:52:17.251098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff5d84-a996-4b58-9377-820c5254cba3 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:17.252889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 35/71] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:17 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:17.254328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c054543-b363-4451-b052-809d55a4117f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:17.259269 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6db0b-06ca-4493-9e27-9134b6f0f8bf None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:17.259837 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6db0b-06ca-4493-9e27-9134b6f0f8bf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:17.260367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6db0b-06ca-4493-9e27-9134b6f0f8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:17.288227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e7d6db0b-06ca-4493-9e27-9134b6f0f8bf None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:17.289462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e7d6db0b-06ca-4493-9e27-9134b6f0f8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:17.362818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 36/72] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:17.363713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6db0b-06ca-4493-9e27-9134b6f0f8bf None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:17.366121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:17.374795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:17.392345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c054543-b363-4451-b052-809d55a4117f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:17.436176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b10689-d30e-4798-9de5-d04d9c4aa0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:17.436826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10689-d30e-4798-9de5-d04d9c4aa0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:17.437095 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10689-d30e-4798-9de5-d04d9c4aa0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:17.437373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10689-d30e-4798-9de5-d04d9c4aa0d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:17.438116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b10689-d30e-4798-9de5-d04d9c4aa0d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:17.438355 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b10689-d30e-4798-9de5-d04d9c4aa0d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:17.439816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b10689-d30e-4798-9de5-d04d9c4aa0d5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:17.451538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 37/73] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:17.452345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b10689-d30e-4798-9de5-d04d9c4aa0d5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:17.479091 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6db0b-06ca-4493-9e27-9134b6f0f8bf None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:17.532473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c4b65-81fa-4697-9dbb-66c3f29147a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:17.533239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c4b65-81fa-4697-9dbb-66c3f29147a6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:17.533454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c4b65-81fa-4697-9dbb-66c3f29147a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:17.533782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c4b65-81fa-4697-9dbb-66c3f29147a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:17.546026 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c4b65-81fa-4697-9dbb-66c3f29147a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-886c4b65-81fa-4697-9dbb-66c3f29147a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:17.551300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:17.552138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:17.552888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 37/74] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:52:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:17.553950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c4b65-81fa-4697-9dbb-66c3f29147a6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:17.581357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b10689-d30e-4798-9de5-d04d9c4aa0d5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:17.619760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:17.623726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622121f-1f1f-43a5-9be6-df9a3b226feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:17.624533 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622121f-1f1f-43a5-9be6-df9a3b226feb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:17.624967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622121f-1f1f-43a5-9be6-df9a3b226feb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:17.625237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622121f-1f1f-43a5-9be6-df9a3b226feb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:17.637320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622121f-1f1f-43a5-9be6-df9a3b226feb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-f622121f-1f1f-43a5-9be6-df9a3b226feb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:17.643438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:17.644482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 38/75] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:52:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:17.645784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622121f-1f1f-43a5-9be6-df9a3b226feb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:17.657413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:17.667661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c4b65-81fa-4697-9dbb-66c3f29147a6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:17.705750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a28af1-c584-467d-a74e-0dcbcc38e834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:17.706394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a28af1-c584-467d-a74e-0dcbcc38e834 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:17.706682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a28af1-c584-467d-a74e-0dcbcc38e834 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:17.708248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a28af1-c584-467d-a74e-0dcbcc38e834 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:17.724313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a28af1-c584-467d-a74e-0dcbcc38e834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-d5a28af1-c584-467d-a74e-0dcbcc38e834 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:17.729301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:17.730262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 38/76] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:17.735817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a28af1-c584-467d-a74e-0dcbcc38e834 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:17.758074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622121f-1f1f-43a5-9be6-df9a3b226feb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:17.795666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3129b96-e772-4a05-a4e7-feb3cce035fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:17.796305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3129b96-e772-4a05-a4e7-feb3cce035fb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:17.796590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3129b96-e772-4a05-a4e7-feb3cce035fb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:17.796871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3129b96-e772-4a05-a4e7-feb3cce035fb None admin] PATH_INFO: `/v3/users/0487c236de2145229f4d07ed5647f6a2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:17.814137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3129b96-e772-4a05-a4e7-feb3cce035fb None admin] RBAC: Authorizing `identity:get_user(user_id=0487c236de2145229f4d07ed5647f6a2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:17.815644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3129b96-e772-4a05-a4e7-feb3cce035fb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:17.817308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 39/77] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:17 2022] GET /identity/v3/users/0487c236de2145229f4d07ed5647f6a2 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:17.818098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3129b96-e772-4a05-a4e7-feb3cce035fb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:17.839469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a28af1-c584-467d-a74e-0dcbcc38e834 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:17.863283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:17.878667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a45514-2e02-4926-87d1-68489d6ac230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:17.879247 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a45514-2e02-4926-87d1-68489d6ac230 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:17.879522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a45514-2e02-4926-87d1-68489d6ac230 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:17.879794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a45514-2e02-4926-87d1-68489d6ac230 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:17.880469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a45514-2e02-4926-87d1-68489d6ac230 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:17.880771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a45514-2e02-4926-87d1-68489d6ac230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:17.882279 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a45514-2e02-4926-87d1-68489d6ac230 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:17.886397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:17.891280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 39/78] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:17.892097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a45514-2e02-4926-87d1-68489d6ac230 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:17.932302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3129b96-e772-4a05-a4e7-feb3cce035fb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:17.968744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d7e86-5328-427b-b36e-78c2d670edaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:17.969490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d7e86-5328-427b-b36e-78c2d670edaf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:17.969859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d7e86-5328-427b-b36e-78c2d670edaf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:17.970265 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d7e86-5328-427b-b36e-78c2d670edaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:17.971546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d7e86-5328-427b-b36e-78c2d670edaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:17.973128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d7e86-5328-427b-b36e-78c2d670edaf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:17.974884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 40/79] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:17.979856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d7e86-5328-427b-b36e-78c2d670edaf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:17.998939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a45514-2e02-4926-87d1-68489d6ac230 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:18.037191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0bdc6-227f-4210-9f05-19b8ee2be315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:18.037802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bdc6-227f-4210-9f05-19b8ee2be315 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:18.038088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bdc6-227f-4210-9f05-19b8ee2be315 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:18.038365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bdc6-227f-4210-9f05-19b8ee2be315 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:18.039061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0bdc6-227f-4210-9f05-19b8ee2be315 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:18.039374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0bdc6-227f-4210-9f05-19b8ee2be315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:18.040757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0bdc6-227f-4210-9f05-19b8ee2be315 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:18.049629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 40/80] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:52:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:18.050592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0bdc6-227f-4210-9f05-19b8ee2be315 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:18.095275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d7e86-5328-427b-b36e-78c2d670edaf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:18.098919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:18.130183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455cbdc-e3f9-40a0-a695-98d258cd02b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:18.130724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455cbdc-e3f9-40a0-a695-98d258cd02b0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:18.131034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455cbdc-e3f9-40a0-a695-98d258cd02b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:18.131306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455cbdc-e3f9-40a0-a695-98d258cd02b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:18.131957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455cbdc-e3f9-40a0-a695-98d258cd02b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:18.132255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455cbdc-e3f9-40a0-a695-98d258cd02b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:18.133634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455cbdc-e3f9-40a0-a695-98d258cd02b0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:18.134986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:18.142911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 41/81] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:18.143641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455cbdc-e3f9-40a0-a695-98d258cd02b0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:18.159059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0bdc6-227f-4210-9f05-19b8ee2be315 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:18.209337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164105ee-eaf2-4ad5-acd8-3470f2a76c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:18.209959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164105ee-eaf2-4ad5-acd8-3470f2a76c20 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:18.210242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164105ee-eaf2-4ad5-acd8-3470f2a76c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:18.210519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164105ee-eaf2-4ad5-acd8-3470f2a76c20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:18.249916 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455cbdc-e3f9-40a0-a695-98d258cd02b0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:18.279652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164105ee-eaf2-4ad5-acd8-3470f2a76c20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:18.281140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164105ee-eaf2-4ad5-acd8-3470f2a76c20 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:18.286120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b0d5b-7c04-4693-83d1-c72abdb1e4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:18.286757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b0d5b-7c04-4693-83d1-c72abdb1e4ce None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:18.287066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b0d5b-7c04-4693-83d1-c72abdb1e4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:18.287343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b0d5b-7c04-4693-83d1-c72abdb1e4ce None admin] PATH_INFO: `/v3/users/d723a6acb5a24745b0576ef9d3892945` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:18.288486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b0d5b-7c04-4693-83d1-c72abdb1e4ce None admin] RBAC: Authorizing `identity:get_user(user_id=d723a6acb5a24745b0576ef9d3892945)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:18.289943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b0d5b-7c04-4693-83d1-c72abdb1e4ce None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:18.291557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 42/82] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:18 2022] GET /identity/v3/users/d723a6acb5a24745b0576ef9d3892945 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:18.292311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b0d5b-7c04-4693-83d1-c72abdb1e4ce None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:18.331724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:18.355579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:18.388692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 41/83] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:18.389372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164105ee-eaf2-4ad5-acd8-3470f2a76c20 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:18.396135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b0d5b-7c04-4693-83d1-c72abdb1e4ce None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:18.430688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afb9dc-913c-4f3b-bcbb-5db707828428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:18.431327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afb9dc-913c-4f3b-bcbb-5db707828428 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:18.431603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afb9dc-913c-4f3b-bcbb-5db707828428 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:18.431874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afb9dc-913c-4f3b-bcbb-5db707828428 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:18.432541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afb9dc-913c-4f3b-bcbb-5db707828428 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:18.433971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afb9dc-913c-4f3b-bcbb-5db707828428 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:18.455276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 43/84] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:52:18 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:18.456107 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afb9dc-913c-4f3b-bcbb-5db707828428 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:18.493192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164105ee-eaf2-4ad5-acd8-3470f2a76c20 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:18.540119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2cfce-a21d-4987-8f12-b0e227e104b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:18.540818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2cfce-a21d-4987-8f12-b0e227e104b0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:18.541051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2cfce-a21d-4987-8f12-b0e227e104b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:18.541328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2cfce-a21d-4987-8f12-b0e227e104b0 None admin] PATH_INFO: `/v3/users/0487c236de2145229f4d07ed5647f6a2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:18.542580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2cfce-a21d-4987-8f12-b0e227e104b0 None admin] RBAC: Authorizing `identity:get_user(user_id=0487c236de2145229f4d07ed5647f6a2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:18.544120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2cfce-a21d-4987-8f12-b0e227e104b0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:18.545849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 42/85] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:18 2022] GET /identity/v3/users/0487c236de2145229f4d07ed5647f6a2 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:18.545849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2cfce-a21d-4987-8f12-b0e227e104b0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:18.568842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:18.585676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afb9dc-913c-4f3b-bcbb-5db707828428 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:18.609446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:18.632894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767de5ea-ed67-4181-bff6-6a24dd6eae62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:18.633547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767de5ea-ed67-4181-bff6-6a24dd6eae62 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:18.633860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767de5ea-ed67-4181-bff6-6a24dd6eae62 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:18.634106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767de5ea-ed67-4181-bff6-6a24dd6eae62 None admin] PATH_INFO: `/v3/users/0487c236de2145229f4d07ed5647f6a2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:18.635296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767de5ea-ed67-4181-bff6-6a24dd6eae62 None admin] RBAC: Authorizing `identity:get_user(user_id=0487c236de2145229f4d07ed5647f6a2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:18.636757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767de5ea-ed67-4181-bff6-6a24dd6eae62 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:18.638993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 44/86] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:18 2022] GET /identity/v3/users/0487c236de2145229f4d07ed5647f6a2 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:18.639721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767de5ea-ed67-4181-bff6-6a24dd6eae62 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:18.687171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2cfce-a21d-4987-8f12-b0e227e104b0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:18.727919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35f7cb-ead3-42c1-a276-e53923ec213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:18.728598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35f7cb-ead3-42c1-a276-e53923ec213f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:18.728840 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35f7cb-ead3-42c1-a276-e53923ec213f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:18.729116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35f7cb-ead3-42c1-a276-e53923ec213f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:18.781040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767de5ea-ed67-4181-bff6-6a24dd6eae62 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:18.823721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35f7cb-ead3-42c1-a276-e53923ec213f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:18.825802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35f7cb-ead3-42c1-a276-e53923ec213f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:18.828195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b113144-4b1d-42f1-a80b-2a4420332472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:18.828721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b113144-4b1d-42f1-a80b-2a4420332472 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:18.829009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b113144-4b1d-42f1-a80b-2a4420332472 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:18.829289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b113144-4b1d-42f1-a80b-2a4420332472 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:18.830468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b113144-4b1d-42f1-a80b-2a4420332472 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:18.831956 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b113144-4b1d-42f1-a80b-2a4420332472 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:18.833847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 45/87] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:18 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:18.834422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b113144-4b1d-42f1-a80b-2a4420332472 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:18.846320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:18.915771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:18.957875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 43/88] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:18.958990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35f7cb-ead3-42c1-a276-e53923ec213f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:18.975986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b113144-4b1d-42f1-a80b-2a4420332472 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:19.023536 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c2309-2468-4ec3-a48b-551b8f785e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:19.024166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c2309-2468-4ec3-a48b-551b8f785e22 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:19.024457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c2309-2468-4ec3-a48b-551b8f785e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:19.025014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c2309-2468-4ec3-a48b-551b8f785e22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:19.069140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35f7cb-ead3-42c1-a276-e53923ec213f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:19.085476 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:19.107678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ac8e4-f9cf-472f-bb78-5ff1e8ff3528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:19.108413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ac8e4-f9cf-472f-bb78-5ff1e8ff3528 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:19.108765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ac8e4-f9cf-472f-bb78-5ff1e8ff3528 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:19.109106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ac8e4-f9cf-472f-bb78-5ff1e8ff3528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:19.110806 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c2309-2468-4ec3-a48b-551b8f785e22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:19.112444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c2309-2468-4ec3-a48b-551b8f785e22 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:19.179998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:19.183073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ac8e4-f9cf-472f-bb78-5ff1e8ff3528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:19.184573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ac8e4-f9cf-472f-bb78-5ff1e8ff3528 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:19.241712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 46/89] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:19.242421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c2309-2468-4ec3-a48b-551b8f785e22 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:19.330858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 44/90] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:19.331999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ac8e4-f9cf-472f-bb78-5ff1e8ff3528 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:19.354981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:19.375876 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c2309-2468-4ec3-a48b-551b8f785e22 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:19.413928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2fe46-f52d-4b3c-8557-f1591c297c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:19.414548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2fe46-f52d-4b3c-8557-f1591c297c9d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:19.414832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2fe46-f52d-4b3c-8557-f1591c297c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:19.415143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2fe46-f52d-4b3c-8557-f1591c297c9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:19.415878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2fe46-f52d-4b3c-8557-f1591c297c9d None admin] RBAC: Adding query filter params (role.id=34ccf81a9f53404e9edefd846827e5b7, scope.project.id=ba7f09585e054b049eec4280fdd8896c, user.id=0487c236de2145229f4d07ed5647f6a2) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:19.416185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2fe46-f52d-4b3c-8557-f1591c297c9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:19.417624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2fe46-f52d-4b3c-8557-f1591c297c9d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:19.424283 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 47/91] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:19 2022] GET /identity/v3/role_assignments?user.id=0487c236de2145229f4d07ed5647f6a2&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:19.425165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2fe46-f52d-4b3c-8557-f1591c297c9d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:19.472914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:19.509086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ac8e4-f9cf-472f-bb78-5ff1e8ff3528 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:19.587692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0665669-2015-4bce-bf45-05569f2bbd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:19.589182 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0665669-2015-4bce-bf45-05569f2bbd1e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:19.589669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0665669-2015-4bce-bf45-05569f2bbd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:19.590142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0665669-2015-4bce-bf45-05569f2bbd1e None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:19.592534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0665669-2015-4bce-bf45-05569f2bbd1e None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:19.594336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0665669-2015-4bce-bf45-05569f2bbd1e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:19.599097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 45/92] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:19 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:19.601191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0665669-2015-4bce-bf45-05569f2bbd1e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:19.637252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2fe46-f52d-4b3c-8557-f1591c297c9d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:19.700931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:19.704688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa4cad-a1d3-4b19-9a07-db4eadd25dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:19.705671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa4cad-a1d3-4b19-9a07-db4eadd25dda None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:19.705959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa4cad-a1d3-4b19-9a07-db4eadd25dda None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:19.706238 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa4cad-a1d3-4b19-9a07-db4eadd25dda None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:19.707408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa4cad-a1d3-4b19-9a07-db4eadd25dda None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:19.708858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa4cad-a1d3-4b19-9a07-db4eadd25dda None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:19.711594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 48/93] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:19 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:19.713201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa4cad-a1d3-4b19-9a07-db4eadd25dda None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:19.726879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0665669-2015-4bce-bf45-05569f2bbd1e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:19.817270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381f8a2-fe96-4e69-b7bb-e1ec283fb31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:19.817918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381f8a2-fe96-4e69-b7bb-e1ec283fb31e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:19.818197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381f8a2-fe96-4e69-b7bb-e1ec283fb31e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:19.818471 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381f8a2-fe96-4e69-b7bb-e1ec283fb31e None admin] PATH_INFO: `/v3/projects/dc51be6e0fb2485a8d71a49a47ea2fb8` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:19.819867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381f8a2-fe96-4e69-b7bb-e1ec283fb31e None admin] RBAC: Authorizing `identity:get_project(project_id=dc51be6e0fb2485a8d71a49a47ea2fb8)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:19.822049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381f8a2-fe96-4e69-b7bb-e1ec283fb31e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:19.824077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 46/94] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:19 2022] GET /identity/v3/projects/dc51be6e0fb2485a8d71a49a47ea2fb8 => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:19.824855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381f8a2-fe96-4e69-b7bb-e1ec283fb31e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:19.850979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:19.870233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa4cad-a1d3-4b19-9a07-db4eadd25dda None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:19.929973 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c55ab-a66b-45b3-b44e-a35b38e689fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:19.930596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c55ab-a66b-45b3-b44e-a35b38e689fc None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:19.930876 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c55ab-a66b-45b3-b44e-a35b38e689fc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:19.931209 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c55ab-a66b-45b3-b44e-a35b38e689fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:19.935376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c55ab-a66b-45b3-b44e-a35b38e689fc None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=ba7f09585e054b049eec4280fdd8896c, user.id=d723a6acb5a24745b0576ef9d3892945) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:19.935693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c55ab-a66b-45b3-b44e-a35b38e689fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:19.937162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c55ab-a66b-45b3-b44e-a35b38e689fc None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:19.946975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381f8a2-fe96-4e69-b7bb-e1ec283fb31e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:19.952645 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 49/95] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:19 2022] GET /identity/v3/role_assignments?user.id=d723a6acb5a24745b0576ef9d3892945&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:19.954603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c55ab-a66b-45b3-b44e-a35b38e689fc None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:20.046752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76cd39-177b-4103-adae-fd6cfd7ca6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:20.047790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76cd39-177b-4103-adae-fd6cfd7ca6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:20.048524 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76cd39-177b-4103-adae-fd6cfd7ca6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:20.049030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76cd39-177b-4103-adae-fd6cfd7ca6a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:20.050435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76cd39-177b-4103-adae-fd6cfd7ca6a0 None admin] RBAC: Adding query filter params (role.id=372b98bd7cdc4f7fbd4b812a50380cef, scope.project.id=ba7f09585e054b049eec4280fdd8896c, user.id=0487c236de2145229f4d07ed5647f6a2) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:20.051023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76cd39-177b-4103-adae-fd6cfd7ca6a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:20.053252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76cd39-177b-4103-adae-fd6cfd7ca6a0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:20.060570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 47/96] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:19 2022] GET /identity/v3/role_assignments?user.id=0487c236de2145229f4d07ed5647f6a2&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=372b98bd7cdc4f7fbd4b812a50380cef => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:20.124018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c55ab-a66b-45b3-b44e-a35b38e689fc None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:20.176229 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a950dc2-d0d4-49d1-b708-0c52bda9d1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:20.176871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a950dc2-d0d4-49d1-b708-0c52bda9d1e4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:20.177157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a950dc2-d0d4-49d1-b708-0c52bda9d1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:20.177439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a950dc2-d0d4-49d1-b708-0c52bda9d1e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:20.178199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a950dc2-d0d4-49d1-b708-0c52bda9d1e4 None admin] RBAC: Adding query filter params (role.id=34ccf81a9f53404e9edefd846827e5b7, scope.project.id=dc51be6e0fb2485a8d71a49a47ea2fb8, user.id=0487c236de2145229f4d07ed5647f6a2) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:20.178509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a950dc2-d0d4-49d1-b708-0c52bda9d1e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:20.180055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a950dc2-d0d4-49d1-b708-0c52bda9d1e4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:20.192183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 50/97] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:19 2022] GET /identity/v3/role_assignments?user.id=0487c236de2145229f4d07ed5647f6a2&scope.project.id=dc51be6e0fb2485a8d71a49a47ea2fb8&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:20.971083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b0aa-6265-465e-b408-3d0cf7f34faf None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:20.972087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b0aa-6265-465e-b408-3d0cf7f34faf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:20.972594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b0aa-6265-465e-b408-3d0cf7f34faf None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:20.974125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 48/98] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:20.979767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b83e77-5419-4734-9d1a-7aa28f729043 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:20.980688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b83e77-5419-4734-9d1a-7aa28f729043 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:20.981184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b83e77-5419-4734-9d1a-7aa28f729043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:21.007431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b2b83e77-5419-4734-9d1a-7aa28f729043 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:21.008985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b2b83e77-5419-4734-9d1a-7aa28f729043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:21.026279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:21.114098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 51/99] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:21.498684 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea9d92-0663-450a-936a-1894e20d7f5f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:21.499414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea9d92-0663-450a-936a-1894e20d7f5f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:21.499959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea9d92-0663-450a-936a-1894e20d7f5f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:21.501505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 49/100] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:21.507396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d023a04-ff9f-4603-8337-182ddd801933 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:21.507683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d023a04-ff9f-4603-8337-182ddd801933 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:21.507962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d023a04-ff9f-4603-8337-182ddd801933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:21.535010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1d023a04-ff9f-4603-8337-182ddd801933 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:21.536203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1d023a04-ff9f-4603-8337-182ddd801933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:21.622180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 52/101] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:21.679436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea9d92-0663-450a-936a-1894e20d7f5f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:21.703495 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:21.814273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea9d92-0663-450a-936a-1894e20d7f5f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:21.858022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31206027-b6b6-4523-80d1-9960dfa8e536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:21.858659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31206027-b6b6-4523-80d1-9960dfa8e536 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:21.858937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31206027-b6b6-4523-80d1-9960dfa8e536 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:21.859240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31206027-b6b6-4523-80d1-9960dfa8e536 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:21.879841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31206027-b6b6-4523-80d1-9960dfa8e536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-31206027-b6b6-4523-80d1-9960dfa8e536 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:21.885307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:21.886156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:21.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 50/102] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:21.892263 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d023a04-ff9f-4603-8337-182ddd801933 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:22.000137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:22.027170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d023a04-ff9f-4603-8337-182ddd801933 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:22.074474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0192a-59c3-4e4a-a78d-a283743d6578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:22.075503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0192a-59c3-4e4a-a78d-a283743d6578 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:22.076683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0192a-59c3-4e4a-a78d-a283743d6578 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:22.077159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0192a-59c3-4e4a-a78d-a283743d6578 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:22.078024 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0192a-59c3-4e4a-a78d-a283743d6578 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:22.078581 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0192a-59c3-4e4a-a78d-a283743d6578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:22.080186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0192a-59c3-4e4a-a78d-a283743d6578 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:22.089257 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 53/103] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:22.095060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31206027-b6b6-4523-80d1-9960dfa8e536 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:22.198623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0192a-59c3-4e4a-a78d-a283743d6578 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:22.220489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:22.266434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31206027-b6b6-4523-80d1-9960dfa8e536 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:22.335512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0192a-59c3-4e4a-a78d-a283743d6578 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:22.360668 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cafca6-14ca-414e-ab24-1c317279d061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:22.361331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cafca6-14ca-414e-ab24-1c317279d061 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:22.361613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cafca6-14ca-414e-ab24-1c317279d061 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:22.361891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cafca6-14ca-414e-ab24-1c317279d061 None admin] PATH_INFO: `/v3/users/744758ab7db74429aa3497d154d10b77` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:22.372151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74e09c-1930-44f8-99ba-dc3a1e6c20b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:22.372879 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74e09c-1930-44f8-99ba-dc3a1e6c20b2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:22.373165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74e09c-1930-44f8-99ba-dc3a1e6c20b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:22.373445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74e09c-1930-44f8-99ba-dc3a1e6c20b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:22.374609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74e09c-1930-44f8-99ba-dc3a1e6c20b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:22.376132 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74e09c-1930-44f8-99ba-dc3a1e6c20b2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:22.377724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 54/104] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:22.383242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74e09c-1930-44f8-99ba-dc3a1e6c20b2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:22.392634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cafca6-14ca-414e-ab24-1c317279d061 None admin] RBAC: Authorizing `identity:get_user(user_id=744758ab7db74429aa3497d154d10b77)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:22.393892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:22.394296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cafca6-14ca-414e-ab24-1c317279d061 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:22.396518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 51/105] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:22 2022] GET /identity/v3/users/744758ab7db74429aa3497d154d10b77 => generated 289 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:22.406328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cafca6-14ca-414e-ab24-1c317279d061 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:22.542018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74e09c-1930-44f8-99ba-dc3a1e6c20b2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:22.620807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cafca6-14ca-414e-ab24-1c317279d061 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:22.621953 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d075fc-d16a-4529-88c9-11f614001074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:22.623357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d075fc-d16a-4529-88c9-11f614001074 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:22.625243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d075fc-d16a-4529-88c9-11f614001074 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:22.625723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d075fc-d16a-4529-88c9-11f614001074 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:22.626639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d075fc-d16a-4529-88c9-11f614001074 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:22.628332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d075fc-d16a-4529-88c9-11f614001074 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:22.668279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 55/106] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:52:22 2022] POST /identity/v3/projects => generated 302 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 16:52:22.677761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:22.686394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768ccd6-fab1-4d63-b008-c77a10f23b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:22.687648 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768ccd6-fab1-4d63-b008-c77a10f23b77 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:22.688160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768ccd6-fab1-4d63-b008-c77a10f23b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:22.688632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768ccd6-fab1-4d63-b008-c77a10f23b77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:22.791786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80276293-ff8d-426b-8cde-eba2669e1be6 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:22.792098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80276293-ff8d-426b-8cde-eba2669e1be6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:22.792352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80276293-ff8d-426b-8cde-eba2669e1be6 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:22.793878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 56/107] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:22.795843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768ccd6-fab1-4d63-b008-c77a10f23b77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:22.798418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768ccd6-fab1-4d63-b008-c77a10f23b77 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:22.805093 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43615e8-34d8-4d48-9aa9-9c44e29319ec None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:22.805398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43615e8-34d8-4d48-9aa9-9c44e29319ec None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:22.805650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43615e8-34d8-4d48-9aa9-9c44e29319ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:22.842975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e43615e8-34d8-4d48-9aa9-9c44e29319ec None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:22.842975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e43615e8-34d8-4d48-9aa9-9c44e29319ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:22.927615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:22.960915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 57/108] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:22.978972 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 52/109] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 16:52:22.979156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43615e8-34d8-4d48-9aa9-9c44e29319ec None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:23.088493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43615e8-34d8-4d48-9aa9-9c44e29319ec None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:23.131122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d94da2-ad2c-42d5-b265-07e1bebda2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:23.131795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94da2-ad2c-42d5-b265-07e1bebda2bd None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:23.132081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94da2-ad2c-42d5-b265-07e1bebda2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:23.132359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94da2-ad2c-42d5-b265-07e1bebda2bd None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:23.133674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d94da2-ad2c-42d5-b265-07e1bebda2bd None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:23.135296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d94da2-ad2c-42d5-b265-07e1bebda2bd None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:23.137422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 58/110] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:22 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:23.144502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768ccd6-fab1-4d63-b008-c77a10f23b77 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:23.193301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:23.253656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284d09-ba4b-4189-9dc6-1c5408436d0f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:23.254286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284d09-ba4b-4189-9dc6-1c5408436d0f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:23.254763 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284d09-ba4b-4189-9dc6-1c5408436d0f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:23.256796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 59/111] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:23.262741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6bd6f-d0ae-4f95-beaa-62fc01b6d5a0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:23.263555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6bd6f-d0ae-4f95-beaa-62fc01b6d5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:23.264035 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6bd6f-d0ae-4f95-beaa-62fc01b6d5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:23.284163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768ccd6-fab1-4d63-b008-c77a10f23b77 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:23.300041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-74b6bd6f-d0ae-4f95-beaa-62fc01b6d5a0 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:23.301259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-74b6bd6f-d0ae-4f95-beaa-62fc01b6d5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:23.334578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0d190-0096-4eea-928b-112f6d4e1443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:23.335245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0d190-0096-4eea-928b-112f6d4e1443 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:23.335528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0d190-0096-4eea-928b-112f6d4e1443 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:23.335801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0d190-0096-4eea-928b-112f6d4e1443 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:23.336560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0d190-0096-4eea-928b-112f6d4e1443 None admin] RBAC: Adding query filter params (role.id=0ab8ef19a25b4e489c9abb312698599c, scope.project.id=ba7f09585e054b049eec4280fdd8896c, user.id=744758ab7db74429aa3497d154d10b77) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:23.336963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0d190-0096-4eea-928b-112f6d4e1443 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:23.338673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0d190-0096-4eea-928b-112f6d4e1443 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:23.350403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 53/112] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:23 2022] GET /identity/v3/role_assignments?user.id=744758ab7db74429aa3497d154d10b77&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=0ab8ef19a25b4e489c9abb312698599c => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:23.355365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0d190-0096-4eea-928b-112f6d4e1443 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:23.384460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:23.404875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 60/113] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:23.504141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:23.511247 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0d190-0096-4eea-928b-112f6d4e1443 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:23.570317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f1647-97cd-4d76-ae95-309831fa0f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:23.570994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f1647-97cd-4d76-ae95-309831fa0f6a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:23.571279 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f1647-97cd-4d76-ae95-309831fa0f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:23.571545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f1647-97cd-4d76-ae95-309831fa0f6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:23.589805 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f1647-97cd-4d76-ae95-309831fa0f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-a63f1647-97cd-4d76-ae95-309831fa0f6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:23.596217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:23.597162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:23.597915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:23.603763 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6bd6f-d0ae-4f95-beaa-62fc01b6d5a0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:23.607719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 54/114] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:23.746632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627df9f-55eb-4d1a-85d9-9e9de789c0de None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:23.747014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627df9f-55eb-4d1a-85d9-9e9de789c0de None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:23.747211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627df9f-55eb-4d1a-85d9-9e9de789c0de None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:23.751880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 55/115] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:23.766938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d46a0-2899-41ef-9fc2-5f12027be384 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:23.767109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6bd6f-d0ae-4f95-beaa-62fc01b6d5a0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:23.767631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d46a0-2899-41ef-9fc2-5f12027be384 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:23.768250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d46a0-2899-41ef-9fc2-5f12027be384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:23.800057 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8c5d46a0-2899-41ef-9fc2-5f12027be384 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:23.802469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8c5d46a0-2899-41ef-9fc2-5f12027be384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:23.832634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96d631-503a-4419-ac7e-96a196ca227b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:23.833277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96d631-503a-4419-ac7e-96a196ca227b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:23.833548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96d631-503a-4419-ac7e-96a196ca227b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:23.833826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96d631-503a-4419-ac7e-96a196ca227b None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:23.834501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96d631-503a-4419-ac7e-96a196ca227b None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:23.834803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96d631-503a-4419-ac7e-96a196ca227b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:23.836220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96d631-503a-4419-ac7e-96a196ca227b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:23.853757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 61/116] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:23.862025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96d631-503a-4419-ac7e-96a196ca227b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:23.884795 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 56/117] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:23.886049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d46a0-2899-41ef-9fc2-5f12027be384 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:23.973964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:24.046077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:24.062145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d46a0-2899-41ef-9fc2-5f12027be384 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:24.087561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96d631-503a-4419-ac7e-96a196ca227b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:24.102787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cdd23-6310-453b-a067-ee3fa74bc2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:24.103454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cdd23-6310-453b-a067-ee3fa74bc2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:24.103733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cdd23-6310-453b-a067-ee3fa74bc2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:24.104006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cdd23-6310-453b-a067-ee3fa74bc2f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:24.116256 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cdd23-6310-453b-a067-ee3fa74bc2f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c55cdd23-6310-453b-a067-ee3fa74bc2f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:24.122020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:24.123702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:24.124692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 57/118] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:52:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:24.125790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8d928-0434-4d3b-a848-294875447eec None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:24.125790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8d928-0434-4d3b-a848-294875447eec None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:24.126064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8d928-0434-4d3b-a848-294875447eec None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:24.132861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 58/119] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:24.133844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8d928-0434-4d3b-a848-294875447eec None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:24.168618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc583af-4354-490a-af3a-104e3ec71895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:24.170886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc583af-4354-490a-af3a-104e3ec71895 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:24.171436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc583af-4354-490a-af3a-104e3ec71895 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:24.172207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc583af-4354-490a-af3a-104e3ec71895 None admin] PATH_INFO: `/v3/users/0487c236de2145229f4d07ed5647f6a2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:24.173910 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc583af-4354-490a-af3a-104e3ec71895 None admin] RBAC: Authorizing `identity:get_user(user_id=0487c236de2145229f4d07ed5647f6a2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:24.175659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc583af-4354-490a-af3a-104e3ec71895 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:24.177718 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 62/120] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:23 2022] GET /identity/v3/users/0487c236de2145229f4d07ed5647f6a2 => generated 275 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:24.179862 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22c1e8-6bca-4658-9f9c-8f79f39b16a0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:24.180358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22c1e8-6bca-4658-9f9c-8f79f39b16a0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:24.180854 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22c1e8-6bca-4658-9f9c-8f79f39b16a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:24.205732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1e22c1e8-6bca-4658-9f9c-8f79f39b16a0 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:24.207480 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1e22c1e8-6bca-4658-9f9c-8f79f39b16a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:24.273829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8d928-0434-4d3b-a848-294875447eec None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:24.295119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 63/121] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:24.296072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22c1e8-6bca-4658-9f9c-8f79f39b16a0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:24.327222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b3187-008f-4377-b6af-3c7f41c20cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:24.327948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b3187-008f-4377-b6af-3c7f41c20cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:24.328235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b3187-008f-4377-b6af-3c7f41c20cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:24.328565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b3187-008f-4377-b6af-3c7f41c20cb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:24.329370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b3187-008f-4377-b6af-3c7f41c20cb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:24.329680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b3187-008f-4377-b6af-3c7f41c20cb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:24.331205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b3187-008f-4377-b6af-3c7f41c20cb4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:24.342811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 59/122] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:52:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:24.348291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b3187-008f-4377-b6af-3c7f41c20cb4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:24.367451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:24.391080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:24.451724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22c1e8-6bca-4658-9f9c-8f79f39b16a0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:24.501842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979223f-86d8-4ea2-a638-63fefe1379ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:24.502502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979223f-86d8-4ea2-a638-63fefe1379ff None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:24.502786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979223f-86d8-4ea2-a638-63fefe1379ff None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:24.503139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979223f-86d8-4ea2-a638-63fefe1379ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:24.525021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979223f-86d8-4ea2-a638-63fefe1379ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:24.529380 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b3187-008f-4377-b6af-3c7f41c20cb4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-3979223f-86d8-4ea2-a638-63fefe1379ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:24.530398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:24.531358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:24.532112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:24.535724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 64/123] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:52:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:24.541096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979223f-86d8-4ea2-a638-63fefe1379ff None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:24.597297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b917f-03c0-4265-8863-0c80aa177e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:24.598275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b917f-03c0-4265-8863-0c80aa177e13 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:24.599319 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b917f-03c0-4265-8863-0c80aa177e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:24.601120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b917f-03c0-4265-8863-0c80aa177e13 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:24.603437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b917f-03c0-4265-8863-0c80aa177e13 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:24.605100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b917f-03c0-4265-8863-0c80aa177e13 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:24.613085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 60/124] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:24 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:24.614201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b917f-03c0-4265-8863-0c80aa177e13 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:24.711276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979223f-86d8-4ea2-a638-63fefe1379ff None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:24.720190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:24.721140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:24.757389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b917f-03c0-4265-8863-0c80aa177e13 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:24.769090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d00cd-02d8-4de7-b947-7591b5639477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:24.770041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d00cd-02d8-4de7-b947-7591b5639477 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:24.770864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d00cd-02d8-4de7-b947-7591b5639477 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:24.771345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d00cd-02d8-4de7-b947-7591b5639477 None admin] PATH_INFO: `/v3/users/0487c236de2145229f4d07ed5647f6a2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:24.773302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d00cd-02d8-4de7-b947-7591b5639477 None admin] RBAC: Authorizing `identity:get_user(user_id=0487c236de2145229f4d07ed5647f6a2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:24.775153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d00cd-02d8-4de7-b947-7591b5639477 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:24.777586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 65/125] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:24 2022] GET /identity/v3/users/0487c236de2145229f4d07ed5647f6a2 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:24.779315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d00cd-02d8-4de7-b947-7591b5639477 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:24.798438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741efb9-3b36-48cf-b962-d4f7f13fc09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:24.799234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741efb9-3b36-48cf-b962-d4f7f13fc09f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:24.799591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741efb9-3b36-48cf-b962-d4f7f13fc09f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:24.799931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741efb9-3b36-48cf-b962-d4f7f13fc09f None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:24.800687 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741efb9-3b36-48cf-b962-d4f7f13fc09f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:24.801055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741efb9-3b36-48cf-b962-d4f7f13fc09f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:24.802561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741efb9-3b36-48cf-b962-d4f7f13fc09f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:24.811713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 61/126] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:24.812626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741efb9-3b36-48cf-b962-d4f7f13fc09f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:24.912997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d00cd-02d8-4de7-b947-7591b5639477 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:24.931647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741efb9-3b36-48cf-b962-d4f7f13fc09f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:24.953083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfc246-c050-4fea-8a38-363fd9e8ddac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:24.953589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfc246-c050-4fea-8a38-363fd9e8ddac None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:24.953875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfc246-c050-4fea-8a38-363fd9e8ddac None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:24.954158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfc246-c050-4fea-8a38-363fd9e8ddac None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c/users/0487c236de2145229f4d07ed5647f6a2/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:24.956383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfc246-c050-4fea-8a38-363fd9e8ddac None admin] RBAC: Authorizing `identity:create_grant(project_id=ba7f09585e054b049eec4280fdd8896c, user_id=0487c236de2145229f4d07ed5647f6a2, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:24.958143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfc246-c050-4fea-8a38-363fd9e8ddac None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:24.975661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 66/127] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:52:24 2022] PUT /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c/users/0487c236de2145229f4d07ed5647f6a2/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:52:24.976298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfc246-c050-4fea-8a38-363fd9e8ddac None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:24.977457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49768d0c-63f3-4f2b-9021-d6aaa82ae8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:24.978325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49768d0c-63f3-4f2b-9021-d6aaa82ae8db None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:24.978904 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49768d0c-63f3-4f2b-9021-d6aaa82ae8db None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:24.979400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49768d0c-63f3-4f2b-9021-d6aaa82ae8db None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:24.993075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:24.995307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49768d0c-63f3-4f2b-9021-d6aaa82ae8db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-49768d0c-63f3-4f2b-9021-d6aaa82ae8db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:25.001690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:25.002605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:25.003869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:25.006047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 62/128] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:25.007329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49768d0c-63f3-4f2b-9021-d6aaa82ae8db None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:25.073582 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:25.150398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49768d0c-63f3-4f2b-9021-d6aaa82ae8db None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:25.192153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab0002-7d42-4792-b658-d98a2651ea33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:25.193212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab0002-7d42-4792-b658-d98a2651ea33 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:25.193763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab0002-7d42-4792-b658-d98a2651ea33 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:25.194238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab0002-7d42-4792-b658-d98a2651ea33 None admin] PATH_INFO: `/v3/users/d723a6acb5a24745b0576ef9d3892945` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:25.195745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab0002-7d42-4792-b658-d98a2651ea33 None admin] RBAC: Authorizing `identity:get_user(user_id=d723a6acb5a24745b0576ef9d3892945)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:25.197493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab0002-7d42-4792-b658-d98a2651ea33 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:25.199482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 63/129] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:25 2022] GET /identity/v3/users/d723a6acb5a24745b0576ef9d3892945 => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:25.200478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab0002-7d42-4792-b658-d98a2651ea33 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:25.276247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfc246-c050-4fea-8a38-363fd9e8ddac None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:25.278075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:25.327313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab0002-7d42-4792-b658-d98a2651ea33 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:25.328773 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c23a2-07cd-4710-96b6-d5575d8b7c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:25.329815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c23a2-07cd-4710-96b6-d5575d8b7c72 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:25.330169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c23a2-07cd-4710-96b6-d5575d8b7c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:25.331094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c23a2-07cd-4710-96b6-d5575d8b7c72 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:25.335773 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c23a2-07cd-4710-96b6-d5575d8b7c72 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:25.337944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c23a2-07cd-4710-96b6-d5575d8b7c72 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:25.345025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 67/130] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:24 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:25.345025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c23a2-07cd-4710-96b6-d5575d8b7c72 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:25.412500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf0b5c-bd69-4abb-9975-42c329ef37aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:25.413488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0b5c-bd69-4abb-9975-42c329ef37aa None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:25.413950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0b5c-bd69-4abb-9975-42c329ef37aa None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:25.414448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0b5c-bd69-4abb-9975-42c329ef37aa None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:25.415318 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf0b5c-bd69-4abb-9975-42c329ef37aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:25.415831 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf0b5c-bd69-4abb-9975-42c329ef37aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:25.417414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf0b5c-bd69-4abb-9975-42c329ef37aa None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:25.441115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 64/131] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:25.442030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf0b5c-bd69-4abb-9975-42c329ef37aa None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:25.461142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c23a2-07cd-4710-96b6-d5575d8b7c72 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:25.467349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:25.504413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ddbe7-8a9c-4d1a-9ba2-7a4b509eb788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:25.505064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ddbe7-8a9c-4d1a-9ba2-7a4b509eb788 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:25.505348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ddbe7-8a9c-4d1a-9ba2-7a4b509eb788 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:25.505625 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ddbe7-8a9c-4d1a-9ba2-7a4b509eb788 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:25.506762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ddbe7-8a9c-4d1a-9ba2-7a4b509eb788 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:25.508312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ddbe7-8a9c-4d1a-9ba2-7a4b509eb788 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:25.510577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 68/132] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:25 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:25.511453 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ddbe7-8a9c-4d1a-9ba2-7a4b509eb788 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:25.632262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ddbe7-8a9c-4d1a-9ba2-7a4b509eb788 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:25.678433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fb60c-ed2f-454d-825c-fef684055c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:25.679109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fb60c-ed2f-454d-825c-fef684055c91 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:25.679396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fb60c-ed2f-454d-825c-fef684055c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:25.679674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fb60c-ed2f-454d-825c-fef684055c91 None admin] PATH_INFO: `/v3/users/0487c236de2145229f4d07ed5647f6a2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:25.681563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fb60c-ed2f-454d-825c-fef684055c91 None admin] RBAC: Authorizing `identity:get_user(user_id=0487c236de2145229f4d07ed5647f6a2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:25.683315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fb60c-ed2f-454d-825c-fef684055c91 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:25.686155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 69/133] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:25 2022] GET /identity/v3/users/0487c236de2145229f4d07ed5647f6a2 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:25.686987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fb60c-ed2f-454d-825c-fef684055c91 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:25.687898 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf0b5c-bd69-4abb-9975-42c329ef37aa None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:25.718646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:25.740522 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:25.744170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107adcc-9807-4854-b31e-78d1a5c47bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:25.745203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107adcc-9807-4854-b31e-78d1a5c47bd2 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:25.745691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107adcc-9807-4854-b31e-78d1a5c47bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:25.746167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107adcc-9807-4854-b31e-78d1a5c47bd2 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c/users/0487c236de2145229f4d07ed5647f6a2/roles/372b98bd7cdc4f7fbd4b812a50380cef` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:25.764907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107adcc-9807-4854-b31e-78d1a5c47bd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba7f09585e054b049eec4280fdd8896c, user_id=0487c236de2145229f4d07ed5647f6a2, role_id=372b98bd7cdc4f7fbd4b812a50380cef)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:25.766708 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107adcc-9807-4854-b31e-78d1a5c47bd2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:25.795003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 65/134] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:52:25 2022] PUT /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c/users/0487c236de2145229f4d07ed5647f6a2/roles/372b98bd7cdc4f7fbd4b812a50380cef => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:52:25.796336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107adcc-9807-4854-b31e-78d1a5c47bd2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:25.815349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fb60c-ed2f-454d-825c-fef684055c91 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:25.859598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feade2df-d496-4490-85e1-b4ecd995f452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:25.860286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade2df-d496-4490-85e1-b4ecd995f452 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:25.860580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade2df-d496-4490-85e1-b4ecd995f452 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:25.860858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade2df-d496-4490-85e1-b4ecd995f452 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c/users/d723a6acb5a24745b0576ef9d3892945/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:25.863019 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feade2df-d496-4490-85e1-b4ecd995f452 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba7f09585e054b049eec4280fdd8896c, user_id=d723a6acb5a24745b0576ef9d3892945, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:25.864590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feade2df-d496-4490-85e1-b4ecd995f452 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:25.893494 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 70/135] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:52:25 2022] PUT /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c/users/d723a6acb5a24745b0576ef9d3892945/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:52:25.979965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7835c7-c4a6-4bd5-91ab-4829a257d1fe None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:25.980169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7835c7-c4a6-4bd5-91ab-4829a257d1fe None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:25.980458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7835c7-c4a6-4bd5-91ab-4829a257d1fe None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:25.982383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 71/136] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:25.984496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107adcc-9807-4854-b31e-78d1a5c47bd2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:25.996518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f62f23-db30-4f91-b56d-5a7d87a5c02a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:25.996806 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f62f23-db30-4f91-b56d-5a7d87a5c02a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:25.997100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f62f23-db30-4f91-b56d-5a7d87a5c02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:26.036726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-47f62f23-db30-4f91-b56d-5a7d87a5c02a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:26.038000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-47f62f23-db30-4f91-b56d-5a7d87a5c02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:26.070236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19990f61-6764-4e5f-8e1c-7c791c079268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:26.070897 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19990f61-6764-4e5f-8e1c-7c791c079268 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:26.071207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19990f61-6764-4e5f-8e1c-7c791c079268 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:26.071488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19990f61-6764-4e5f-8e1c-7c791c079268 None admin] PATH_INFO: `/v3/projects/dc51be6e0fb2485a8d71a49a47ea2fb8` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:26.074059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19990f61-6764-4e5f-8e1c-7c791c079268 None admin] RBAC: Authorizing `identity:get_project(project_id=dc51be6e0fb2485a8d71a49a47ea2fb8)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:26.077634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19990f61-6764-4e5f-8e1c-7c791c079268 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:26.078462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 66/137] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:25 2022] GET /identity/v3/projects/dc51be6e0fb2485a8d71a49a47ea2fb8 => generated 312 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:26.086235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19990f61-6764-4e5f-8e1c-7c791c079268 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:26.216965 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:26.264373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 72/138] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:26.273085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19990f61-6764-4e5f-8e1c-7c791c079268 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:26.310395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d595e33-81d4-4e6d-96f3-685ee7580fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:26.311217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595e33-81d4-4e6d-96f3-685ee7580fde None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:26.311638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595e33-81d4-4e6d-96f3-685ee7580fde None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:26.311925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595e33-81d4-4e6d-96f3-685ee7580fde None admin] PATH_INFO: `/v3/projects/dc51be6e0fb2485a8d71a49a47ea2fb8/users/0487c236de2145229f4d07ed5647f6a2/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:26.313952 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d595e33-81d4-4e6d-96f3-685ee7580fde None admin] RBAC: Authorizing `identity:create_grant(project_id=dc51be6e0fb2485a8d71a49a47ea2fb8, user_id=0487c236de2145229f4d07ed5647f6a2, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:26.315464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d595e33-81d4-4e6d-96f3-685ee7580fde None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:26.333147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c4532-fe26-4e1e-a55b-f7485097da48 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:26.333671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c4532-fe26-4e1e-a55b-f7485097da48 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:26.333758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c4532-fe26-4e1e-a55b-f7485097da48 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:26.336464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 73/139] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:26.337481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 67/140] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:52:26 2022] PUT /identity/v3/projects/dc51be6e0fb2485a8d71a49a47ea2fb8/users/0487c236de2145229f4d07ed5647f6a2/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:52:26.354870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6338885-a6dd-46c5-9038-c42ff1a0459d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:26.355206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6338885-a6dd-46c5-9038-c42ff1a0459d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:26.355475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6338885-a6dd-46c5-9038-c42ff1a0459d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:26.384096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f6338885-a6dd-46c5-9038-c42ff1a0459d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:26.385300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f6338885-a6dd-46c5-9038-c42ff1a0459d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:26.418085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:26.476926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 74/141] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:26.814468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d595e33-81d4-4e6d-96f3-685ee7580fde None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:26.896214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6338885-a6dd-46c5-9038-c42ff1a0459d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:26.973876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d595e33-81d4-4e6d-96f3-685ee7580fde None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:27.005511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:27.077464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6338885-a6dd-46c5-9038-c42ff1a0459d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:27.079728 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d1747-c6eb-4798-a470-70474481426f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:27.080398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d1747-c6eb-4798-a470-70474481426f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:27.080680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d1747-c6eb-4798-a470-70474481426f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:27.080954 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d1747-c6eb-4798-a470-70474481426f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:27.083745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d1747-c6eb-4798-a470-70474481426f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:27.085265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d1747-c6eb-4798-a470-70474481426f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:27.091583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 68/142] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:27.096040 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d1747-c6eb-4798-a470-70474481426f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:27.165485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabe029-c7aa-4deb-b32a-86f9e854fcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:27.168288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabe029-c7aa-4deb-b32a-86f9e854fcf8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:27.168288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabe029-c7aa-4deb-b32a-86f9e854fcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:27.168288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabe029-c7aa-4deb-b32a-86f9e854fcf8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:27.194976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabe029-c7aa-4deb-b32a-86f9e854fcf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-acabe029-c7aa-4deb-b32a-86f9e854fcf8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:27.214976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:27.215890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:27.217163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 75/143] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:27.226993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabe029-c7aa-4deb-b32a-86f9e854fcf8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:27.266673 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:27.425722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d1747-c6eb-4798-a470-70474481426f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:27.430257 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabe029-c7aa-4deb-b32a-86f9e854fcf8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:27.505768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae06a6-04ec-4ac5-911c-1b74a3837b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:27.506916 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae06a6-04ec-4ac5-911c-1b74a3837b29 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:27.507525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae06a6-04ec-4ac5-911c-1b74a3837b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:27.508015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae06a6-04ec-4ac5-911c-1b74a3837b29 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:27.508958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae06a6-04ec-4ac5-911c-1b74a3837b29 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:27.509503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae06a6-04ec-4ac5-911c-1b74a3837b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:27.511312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae06a6-04ec-4ac5-911c-1b74a3837b29 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:27.536779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 76/144] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:27.543233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae06a6-04ec-4ac5-911c-1b74a3837b29 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:27.566977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60f2de-b1fb-4471-93b3-cc9a47073292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:27.566977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f2de-b1fb-4471-93b3-cc9a47073292 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:27.566977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f2de-b1fb-4471-93b3-cc9a47073292 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:27.566977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f2de-b1fb-4471-93b3-cc9a47073292 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:27.567387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60f2de-b1fb-4471-93b3-cc9a47073292 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:27.567387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60f2de-b1fb-4471-93b3-cc9a47073292 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:27.624474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 69/145] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:52:27 2022] POST /identity/v3/users => generated 283 bytes in 529 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:27.719443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:27.781103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae06a6-04ec-4ac5-911c-1b74a3837b29 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:27.875831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e58b51-5eb1-40cf-8865-a95d0f42d9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:27.876897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e58b51-5eb1-40cf-8865-a95d0f42d9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:27.877391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e58b51-5eb1-40cf-8865-a95d0f42d9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:27.877866 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e58b51-5eb1-40cf-8865-a95d0f42d9a0 None admin] PATH_INFO: `/v3/users/744758ab7db74429aa3497d154d10b77` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:27.879380 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e58b51-5eb1-40cf-8865-a95d0f42d9a0 None admin] RBAC: Authorizing `identity:get_user(user_id=744758ab7db74429aa3497d154d10b77)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:27.881055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e58b51-5eb1-40cf-8865-a95d0f42d9a0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:27.882885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 77/146] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:27 2022] GET /identity/v3/users/744758ab7db74429aa3497d154d10b77 => generated 289 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:27.889251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60f2de-b1fb-4471-93b3-cc9a47073292 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:27.944892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d10f77-f16e-4f68-ac86-ce024d77ce2f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:27.945499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d10f77-f16e-4f68-ac86-ce024d77ce2f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:27.946004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d10f77-f16e-4f68-ac86-ce024d77ce2f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:27.947733 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 78/147] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:27.955697 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26760f-e145-4cad-986a-8bb2f429b1bb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:27.956848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26760f-e145-4cad-986a-8bb2f429b1bb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:27.957327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26760f-e145-4cad-986a-8bb2f429b1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:27.994485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:28.007515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1b26760f-e145-4cad-986a-8bb2f429b1bb None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:28.009316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1b26760f-e145-4cad-986a-8bb2f429b1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:28.142029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60f2de-b1fb-4471-93b3-cc9a47073292 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:28.239058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5751d37-faf3-4fbd-a0c0-9516e02f87f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:28.239766 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5751d37-faf3-4fbd-a0c0-9516e02f87f6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:28.240048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5751d37-faf3-4fbd-a0c0-9516e02f87f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:28.240318 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5751d37-faf3-4fbd-a0c0-9516e02f87f6 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:28.243444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5751d37-faf3-4fbd-a0c0-9516e02f87f6 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:28.244990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5751d37-faf3-4fbd-a0c0-9516e02f87f6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:28.248111 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 70/148] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:27 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:28.249561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 79/149] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:28.253056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26760f-e145-4cad-986a-8bb2f429b1bb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:28.337422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:28.363385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26760f-e145-4cad-986a-8bb2f429b1bb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:28.402525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c747fc-1a29-43b0-ab4d-8274ee6ebdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:28.403226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c747fc-1a29-43b0-ab4d-8274ee6ebdea None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:28.403502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c747fc-1a29-43b0-ab4d-8274ee6ebdea None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:28.403880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c747fc-1a29-43b0-ab4d-8274ee6ebdea None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c/users/744758ab7db74429aa3497d154d10b77/roles/0ab8ef19a25b4e489c9abb312698599c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:28.408316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c747fc-1a29-43b0-ab4d-8274ee6ebdea None admin] RBAC: Authorizing `identity:create_grant(project_id=ba7f09585e054b049eec4280fdd8896c, user_id=744758ab7db74429aa3497d154d10b77, role_id=0ab8ef19a25b4e489c9abb312698599c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:28.409801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c747fc-1a29-43b0-ab4d-8274ee6ebdea None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:28.433201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 80/150] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:52:28 2022] PUT /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c/users/744758ab7db74429aa3497d154d10b77/roles/0ab8ef19a25b4e489c9abb312698599c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:52:28.661263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5751d37-faf3-4fbd-a0c0-9516e02f87f6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:28.802131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5751d37-faf3-4fbd-a0c0-9516e02f87f6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:28.813723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:28.847239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97403ead-c526-430b-9843-8e0eceab7520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:28.847889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97403ead-c526-430b-9843-8e0eceab7520 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:28.848168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97403ead-c526-430b-9843-8e0eceab7520 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:28.848437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97403ead-c526-430b-9843-8e0eceab7520 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:28.860402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97403ead-c526-430b-9843-8e0eceab7520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-97403ead-c526-430b-9843-8e0eceab7520 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:28.865371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:28.866261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:28.867212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 71/151] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:28.872881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c747fc-1a29-43b0-ab4d-8274ee6ebdea None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:28.990567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c747fc-1a29-43b0-ab4d-8274ee6ebdea None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:29.040518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba871074-240b-4a3f-8670-1bbc5dbfe8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:29.041151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba871074-240b-4a3f-8670-1bbc5dbfe8ca None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:29.041430 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba871074-240b-4a3f-8670-1bbc5dbfe8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:29.041701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba871074-240b-4a3f-8670-1bbc5dbfe8ca None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:29.042381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba871074-240b-4a3f-8670-1bbc5dbfe8ca None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:29.042679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba871074-240b-4a3f-8670-1bbc5dbfe8ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:29.044100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba871074-240b-4a3f-8670-1bbc5dbfe8ca None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:29.051709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:29.056540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 81/152] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:29.062384 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97403ead-c526-430b-9843-8e0eceab7520 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:29.171554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7714831-d96c-491d-be6c-9c952e1437da None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:29.171924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7714831-d96c-491d-be6c-9c952e1437da None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:29.172097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7714831-d96c-491d-be6c-9c952e1437da None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:29.173851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 82/153] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:29.180293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b84408-4744-4115-8c3b-5ae6b133f836 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:29.180580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b84408-4744-4115-8c3b-5ae6b133f836 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:29.180842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b84408-4744-4115-8c3b-5ae6b133f836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:29.219985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f6b84408-4744-4115-8c3b-5ae6b133f836 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:29.221201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f6b84408-4744-4115-8c3b-5ae6b133f836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:29.246373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97403ead-c526-430b-9843-8e0eceab7520 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:29.312387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d8b1c-83e2-44f9-aa0b-c9d41824b692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:29.313063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d8b1c-83e2-44f9-aa0b-c9d41824b692 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:29.313345 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d8b1c-83e2-44f9-aa0b-c9d41824b692 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:29.313613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d8b1c-83e2-44f9-aa0b-c9d41824b692 None admin] PATH_INFO: `/v3/users/0487c236de2145229f4d07ed5647f6a2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:29.314979 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d8b1c-83e2-44f9-aa0b-c9d41824b692 None admin] RBAC: Authorizing `identity:get_user(user_id=0487c236de2145229f4d07ed5647f6a2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:29.316507 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d8b1c-83e2-44f9-aa0b-c9d41824b692 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:29.318578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 72/154] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:29 2022] GET /identity/v3/users/0487c236de2145229f4d07ed5647f6a2 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:29.326924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d8b1c-83e2-44f9-aa0b-c9d41824b692 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:29.399344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:29.437948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 83/155] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:29.474228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabe486-b18c-4f66-adcc-07518e9aaa28 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:29.474531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabe486-b18c-4f66-adcc-07518e9aaa28 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:29.474772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabe486-b18c-4f66-adcc-07518e9aaa28 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:29.476155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 84/156] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:29.489032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcf467-5caa-4c9e-978e-f28620ad512c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:29.489356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcf467-5caa-4c9e-978e-f28620ad512c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:29.489580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcf467-5caa-4c9e-978e-f28620ad512c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:29.507511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d8b1c-83e2-44f9-aa0b-c9d41824b692 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:29.529920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-28fcf467-5caa-4c9e-978e-f28620ad512c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:29.531171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-28fcf467-5caa-4c9e-978e-f28620ad512c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:29.551625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc07dee-9bd4-407b-9e72-fc6c4606c6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:29.552394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc07dee-9bd4-407b-9e72-fc6c4606c6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:29.552681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc07dee-9bd4-407b-9e72-fc6c4606c6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:29.552953 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc07dee-9bd4-407b-9e72-fc6c4606c6d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:29.666399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 85/157] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:29.668864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc07dee-9bd4-407b-9e72-fc6c4606c6d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:29.671037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc07dee-9bd4-407b-9e72-fc6c4606c6d9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:29.719487 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:29.787872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 73/158] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:29.796161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcf467-5caa-4c9e-978e-f28620ad512c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:29.887419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:29.926761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde8562-82df-4bc0-ad8e-35838479ba5a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:29.927090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde8562-82df-4bc0-ad8e-35838479ba5a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:29.927346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde8562-82df-4bc0-ad8e-35838479ba5a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:29.928710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 74/159] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:29.941628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcf467-5caa-4c9e-978e-f28620ad512c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:29.953745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b04cd-ba19-4e69-83a8-7936c81c94e2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:29.954053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b04cd-ba19-4e69-83a8-7936c81c94e2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:29.954302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b04cd-ba19-4e69-83a8-7936c81c94e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:29.984975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c13b04cd-ba19-4e69-83a8-7936c81c94e2 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:29.986196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c13b04cd-ba19-4e69-83a8-7936c81c94e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:30.031322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eff9f-6fb8-4e18-a1d8-93e97c8d3714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:30.031894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eff9f-6fb8-4e18-a1d8-93e97c8d3714 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:30.032172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eff9f-6fb8-4e18-a1d8-93e97c8d3714 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:30.032441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eff9f-6fb8-4e18-a1d8-93e97c8d3714 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:30.033705 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eff9f-6fb8-4e18-a1d8-93e97c8d3714 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:30.035213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eff9f-6fb8-4e18-a1d8-93e97c8d3714 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:30.043713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 86/160] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:29 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:30.044822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eff9f-6fb8-4e18-a1d8-93e97c8d3714 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:30.128314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 75/161] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:30.129210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b04cd-ba19-4e69-83a8-7936c81c94e2 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:30.264281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:30.307210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eff9f-6fb8-4e18-a1d8-93e97c8d3714 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:30.328854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b04cd-ba19-4e69-83a8-7936c81c94e2 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:30.379318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:30.399801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d21f9-6712-49ea-b309-4e769e94104e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:30.400635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea74655-3918-4dfa-9094-4b44fae57d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:30.401080 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d21f9-6712-49ea-b309-4e769e94104e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:30.401279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74655-3918-4dfa-9094-4b44fae57d42 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:30.401562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74655-3918-4dfa-9094-4b44fae57d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:30.401836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74655-3918-4dfa-9094-4b44fae57d42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:30.402214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d21f9-6712-49ea-b309-4e769e94104e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:30.402718 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d21f9-6712-49ea-b309-4e769e94104e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:30.403737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d21f9-6712-49ea-b309-4e769e94104e None admin] RBAC: Adding query filter params (role.id=34ccf81a9f53404e9edefd846827e5b7, scope.project.id=ba7f09585e054b049eec4280fdd8896c, user.id=0487c236de2145229f4d07ed5647f6a2) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:30.404260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d21f9-6712-49ea-b309-4e769e94104e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:30.405940 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d21f9-6712-49ea-b309-4e769e94104e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:30.413518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 76/162] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:30 2022] GET /identity/v3/role_assignments?user.id=0487c236de2145229f4d07ed5647f6a2&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:30.414692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d21f9-6712-49ea-b309-4e769e94104e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:30.427683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea74655-3918-4dfa-9094-4b44fae57d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-cea74655-3918-4dfa-9094-4b44fae57d42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:30.433245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:30.434237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:30.440325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 87/163] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:52:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:30.452639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea74655-3918-4dfa-9094-4b44fae57d42 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:30.566439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d21f9-6712-49ea-b309-4e769e94104e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:30.618660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:30.624052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea74655-3918-4dfa-9094-4b44fae57d42 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:30.631689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aba10b-57b9-4f40-8875-727822f80803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:30.632459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aba10b-57b9-4f40-8875-727822f80803 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:30.632815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aba10b-57b9-4f40-8875-727822f80803 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:30.633164 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aba10b-57b9-4f40-8875-727822f80803 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:30.648676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aba10b-57b9-4f40-8875-727822f80803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f1aba10b-57b9-4f40-8875-727822f80803 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:30.654369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:30.655262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:30.656035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:30.659875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 77/164] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:52:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:30.666084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aba10b-57b9-4f40-8875-727822f80803 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:30.681272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c9c99-9b9e-48de-8fbf-ebb188f58ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:30.681900 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c9c99-9b9e-48de-8fbf-ebb188f58ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:30.682183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c9c99-9b9e-48de-8fbf-ebb188f58ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:30.682458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c9c99-9b9e-48de-8fbf-ebb188f58ce3 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:30.683162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c9c99-9b9e-48de-8fbf-ebb188f58ce3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:30.683468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c9c99-9b9e-48de-8fbf-ebb188f58ce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:30.684974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c9c99-9b9e-48de-8fbf-ebb188f58ce3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:30.698741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 88/165] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:52:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:30.709839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c9c99-9b9e-48de-8fbf-ebb188f58ce3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:30.816291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:30.892554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c9c99-9b9e-48de-8fbf-ebb188f58ce3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:30.897366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aba10b-57b9-4f40-8875-727822f80803 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:30.946731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7efa8b-94b6-4b41-a8f2-13870102c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:30.947413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7efa8b-94b6-4b41-a8f2-13870102c792 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:30.947693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7efa8b-94b6-4b41-a8f2-13870102c792 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:30.947966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7efa8b-94b6-4b41-a8f2-13870102c792 None admin] PATH_INFO: `/v3/users/0487c236de2145229f4d07ed5647f6a2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:30.949125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7efa8b-94b6-4b41-a8f2-13870102c792 None admin] RBAC: Authorizing `identity:get_user(user_id=0487c236de2145229f4d07ed5647f6a2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:30.950645 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7efa8b-94b6-4b41-a8f2-13870102c792 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:30.952748 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 89/166] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:30 2022] GET /identity/v3/users/0487c236de2145229f4d07ed5647f6a2 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:30.953612 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7efa8b-94b6-4b41-a8f2-13870102c792 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:30.989192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb5c68-2132-42d1-a718-a808abb19dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:30.989834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb5c68-2132-42d1-a718-a808abb19dde None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:30.990112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb5c68-2132-42d1-a718-a808abb19dde None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:30.990382 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb5c68-2132-42d1-a718-a808abb19dde None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:30.991085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb5c68-2132-42d1-a718-a808abb19dde None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:30.991385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb5c68-2132-42d1-a718-a808abb19dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:30.992797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb5c68-2132-42d1-a718-a808abb19dde None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:31.007678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 78/167] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:31.008632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb5c68-2132-42d1-a718-a808abb19dde None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:31.077879 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7efa8b-94b6-4b41-a8f2-13870102c792 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:31.111853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:31.118036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fbd88-1e65-4511-afd2-0fffd18db26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:31.118937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fbd88-1e65-4511-afd2-0fffd18db26c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:31.119468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fbd88-1e65-4511-afd2-0fffd18db26c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:31.119791 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fbd88-1e65-4511-afd2-0fffd18db26c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:31.132129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fbd88-1e65-4511-afd2-0fffd18db26c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-209fbd88-1e65-4511-afd2-0fffd18db26c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:31.138981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:31.139962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:31.140765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 90/168] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:31.141423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fbd88-1e65-4511-afd2-0fffd18db26c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:31.142188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:31.262100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fbd88-1e65-4511-afd2-0fffd18db26c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:31.286054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb5c68-2132-42d1-a718-a808abb19dde None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:31.299766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78542f33-1eff-4740-8065-1c39c9857265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:31.300651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78542f33-1eff-4740-8065-1c39c9857265 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:31.301078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78542f33-1eff-4740-8065-1c39c9857265 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:31.301723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78542f33-1eff-4740-8065-1c39c9857265 None admin] PATH_INFO: `/v3/users/d723a6acb5a24745b0576ef9d3892945` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:31.303133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78542f33-1eff-4740-8065-1c39c9857265 None admin] RBAC: Authorizing `identity:get_user(user_id=d723a6acb5a24745b0576ef9d3892945)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:31.304869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78542f33-1eff-4740-8065-1c39c9857265 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:31.306619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 91/169] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:31 2022] GET /identity/v3/users/d723a6acb5a24745b0576ef9d3892945 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:31.307552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78542f33-1eff-4740-8065-1c39c9857265 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:31.369343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:31.383530 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cf35e-8469-4f5f-8b99-0a338f181f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:31.384327 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cf35e-8469-4f5f-8b99-0a338f181f7a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:31.384706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cf35e-8469-4f5f-8b99-0a338f181f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:31.385084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cf35e-8469-4f5f-8b99-0a338f181f7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:31.478064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78542f33-1eff-4740-8065-1c39c9857265 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:31.547202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cf35e-8469-4f5f-8b99-0a338f181f7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:31.549006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cf35e-8469-4f5f-8b99-0a338f181f7a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:31.560650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26933b-de5e-4f76-8893-c0998136e052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:31.561290 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26933b-de5e-4f76-8893-c0998136e052 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:31.561610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26933b-de5e-4f76-8893-c0998136e052 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:31.561906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26933b-de5e-4f76-8893-c0998136e052 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:31.562573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26933b-de5e-4f76-8893-c0998136e052 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:31.562870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26933b-de5e-4f76-8893-c0998136e052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:31.564287 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26933b-de5e-4f76-8893-c0998136e052 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:31.582726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 92/170] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:31.583670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26933b-de5e-4f76-8893-c0998136e052 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:31.663820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:31.787534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 79/171] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 779 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:31.789737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cf35e-8469-4f5f-8b99-0a338f181f7a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:31.807823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26933b-de5e-4f76-8893-c0998136e052 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:31.826529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:31.883944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb18aad-cdb7-4e72-9004-ca836f21b81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:31.884960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb18aad-cdb7-4e72-9004-ca836f21b81c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:31.885441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb18aad-cdb7-4e72-9004-ca836f21b81c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:31.887473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb18aad-cdb7-4e72-9004-ca836f21b81c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:31.992778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb18aad-cdb7-4e72-9004-ca836f21b81c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:31.994677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb18aad-cdb7-4e72-9004-ca836f21b81c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:31.998486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cf35e-8469-4f5f-8b99-0a338f181f7a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:32.077667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93838dce-63c0-4e9f-951a-a121e68f8534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:32.078230 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838dce-63c0-4e9f-951a-a121e68f8534 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:32.078505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838dce-63c0-4e9f-951a-a121e68f8534 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:32.078772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838dce-63c0-4e9f-951a-a121e68f8534 None admin] PATH_INFO: `/v3/users/0487c236de2145229f4d07ed5647f6a2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:32.083778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93838dce-63c0-4e9f-951a-a121e68f8534 None admin] RBAC: Authorizing `identity:get_user(user_id=0487c236de2145229f4d07ed5647f6a2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:32.085334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93838dce-63c0-4e9f-951a-a121e68f8534 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:32.087007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 80/172] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:31 2022] GET /identity/v3/users/0487c236de2145229f4d07ed5647f6a2 => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:32.087785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93838dce-63c0-4e9f-951a-a121e68f8534 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:32.110566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 93/173] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:32.111878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb18aad-cdb7-4e72-9004-ca836f21b81c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:32.124996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:32.132099 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:32.237017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb18aad-cdb7-4e72-9004-ca836f21b81c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:32.276309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d156d-b1c3-4e2b-b4e9-a88cb0529833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:32.277302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d156d-b1c3-4e2b-b4e9-a88cb0529833 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:32.277803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d156d-b1c3-4e2b-b4e9-a88cb0529833 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:32.278277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d156d-b1c3-4e2b-b4e9-a88cb0529833 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:32.303007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93838dce-63c0-4e9f-951a-a121e68f8534 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:32.383249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e4577-82a7-40c2-a2a7-5726ee8d6a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:32.383914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e4577-82a7-40c2-a2a7-5726ee8d6a61 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:32.384351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e4577-82a7-40c2-a2a7-5726ee8d6a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:32.384766 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e4577-82a7-40c2-a2a7-5726ee8d6a61 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:32.387261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e4577-82a7-40c2-a2a7-5726ee8d6a61 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:32.389167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e4577-82a7-40c2-a2a7-5726ee8d6a61 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:32.392181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 81/174] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:32 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:32.393259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e4577-82a7-40c2-a2a7-5726ee8d6a61 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:32.395892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d156d-b1c3-4e2b-b4e9-a88cb0529833 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:32.397643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d156d-b1c3-4e2b-b4e9-a88cb0529833 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:32.411411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:32.572457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 94/175] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:32.573860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d156d-b1c3-4e2b-b4e9-a88cb0529833 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:32.602489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:32.629694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e4577-82a7-40c2-a2a7-5726ee8d6a61 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:32.691571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d156d-b1c3-4e2b-b4e9-a88cb0529833 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:32.708690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:32.731911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86985497-687d-475f-acee-9171a89602fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:32.732263 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185e307-6a6f-4056-a43c-3ee33aa35b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:32.732591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985497-687d-475f-acee-9171a89602fa None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:32.732870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985497-687d-475f-acee-9171a89602fa None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:32.733146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985497-687d-475f-acee-9171a89602fa None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:32.733660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185e307-6a6f-4056-a43c-3ee33aa35b0a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:32.734085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185e307-6a6f-4056-a43c-3ee33aa35b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:32.734498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185e307-6a6f-4056-a43c-3ee33aa35b0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:32.735457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185e307-6a6f-4056-a43c-3ee33aa35b0a None admin] RBAC: Adding query filter params (role.id=372b98bd7cdc4f7fbd4b812a50380cef, scope.project.id=ba7f09585e054b049eec4280fdd8896c, user.id=0487c236de2145229f4d07ed5647f6a2) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:32.736063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185e307-6a6f-4056-a43c-3ee33aa35b0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:32.737831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185e307-6a6f-4056-a43c-3ee33aa35b0a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:32.739460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86985497-687d-475f-acee-9171a89602fa None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:32.741016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86985497-687d-475f-acee-9171a89602fa None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:32.743124 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 82/176] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:32 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:32.744738 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86157c56-fd53-4dbb-aa37-d1b89eb51794 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:32.745245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86157c56-fd53-4dbb-aa37-d1b89eb51794 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:32.745687 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86157c56-fd53-4dbb-aa37-d1b89eb51794 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:32.746411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 95/177] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:32 2022] GET /identity/v3/role_assignments?user.id=0487c236de2145229f4d07ed5647f6a2&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=372b98bd7cdc4f7fbd4b812a50380cef => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:32.747629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 83/178] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:32.749073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc1940-9f76-453a-beba-b1c3ad111c6f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:32.749577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc1940-9f76-453a-beba-b1c3ad111c6f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:32.750027 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc1940-9f76-453a-beba-b1c3ad111c6f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:32.751654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 84/179] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:32.752909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185e307-6a6f-4056-a43c-3ee33aa35b0a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:32.759850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc1940-9f76-453a-beba-b1c3ad111c6f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:32.956348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185e307-6a6f-4056-a43c-3ee33aa35b0a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:32.961386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc1940-9f76-453a-beba-b1c3ad111c6f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:33.003518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb324c7-435c-46e6-babf-bba80928d710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:33.004236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb324c7-435c-46e6-babf-bba80928d710 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:33.004564 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb324c7-435c-46e6-babf-bba80928d710 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:33.004796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb324c7-435c-46e6-babf-bba80928d710 None admin] PATH_INFO: `/v3/projects/dc51be6e0fb2485a8d71a49a47ea2fb8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:33.005978 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb324c7-435c-46e6-babf-bba80928d710 None admin] RBAC: Authorizing `identity:get_project(project_id=dc51be6e0fb2485a8d71a49a47ea2fb8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:33.008387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb324c7-435c-46e6-babf-bba80928d710 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:33.010403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 96/180] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:32 2022] GET /identity/v3/projects/dc51be6e0fb2485a8d71a49a47ea2fb8 => generated 312 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:33.011693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e63e4-0f65-4fbb-b49c-46679583fff9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:33.012045 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e63e4-0f65-4fbb-b49c-46679583fff9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:33.012324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e63e4-0f65-4fbb-b49c-46679583fff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:33.024746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b387e23-994e-4b66-b39c-030a0f14703b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:33.025729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b387e23-994e-4b66-b39c-030a0f14703b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:33.026183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b387e23-994e-4b66-b39c-030a0f14703b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:33.026620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b387e23-994e-4b66-b39c-030a0f14703b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:33.029327 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b387e23-994e-4b66-b39c-030a0f14703b None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=ba7f09585e054b049eec4280fdd8896c, user.id=d723a6acb5a24745b0576ef9d3892945) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:33.030030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b387e23-994e-4b66-b39c-030a0f14703b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:33.032420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b387e23-994e-4b66-b39c-030a0f14703b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:33.033725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-530e63e4-0f65-4fbb-b49c-46679583fff9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:33.034940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-530e63e4-0f65-4fbb-b49c-46679583fff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:33.044895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 85/181] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:32 2022] GET /identity/v3/role_assignments?user.id=d723a6acb5a24745b0576ef9d3892945&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:33.046782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b2043-8daa-449f-a86b-6bbbea0e404c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:33.055462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b2043-8daa-449f-a86b-6bbbea0e404c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:33.055992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b2043-8daa-449f-a86b-6bbbea0e404c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:33.079998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:33.106913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7e1b2043-8daa-449f-a86b-6bbbea0e404c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:33.108824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7e1b2043-8daa-449f-a86b-6bbbea0e404c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:33.125368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 97/182] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:33.131094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bee90-7d44-4e0d-95f1-fd962e48b25d None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:33.131501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bee90-7d44-4e0d-95f1-fd962e48b25d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:33.131788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bee90-7d44-4e0d-95f1-fd962e48b25d None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:33.132893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:33.133295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 98/183] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:33.138360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bee90-7d44-4e0d-95f1-fd962e48b25d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:33.212576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 86/184] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:33.214678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7853cb5-d096-4e1c-9251-249e8534fa16 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:33.219673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7853cb5-d096-4e1c-9251-249e8534fa16 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:33.220931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7853cb5-d096-4e1c-9251-249e8534fa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:33.255919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b7853cb5-d096-4e1c-9251-249e8534fa16 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:33.257868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b7853cb5-d096-4e1c-9251-249e8534fa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:33.379626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bee90-7d44-4e0d-95f1-fd962e48b25d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:33.389650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 87/185] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:33.391102 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91d84d-5eab-493c-bf92-606b122ab023 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:33.399013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91d84d-5eab-493c-bf92-606b122ab023 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:33.399013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91d84d-5eab-493c-bf92-606b122ab023 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:33.399013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 88/186] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:33.402419 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117516f-37e7-4784-b3ad-945ee6685235 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:33.402708 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117516f-37e7-4784-b3ad-945ee6685235 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:33.403003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117516f-37e7-4784-b3ad-945ee6685235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:33.440894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0117516f-37e7-4784-b3ad-945ee6685235 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:33.442115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0117516f-37e7-4784-b3ad-945ee6685235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:33.446689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af39689-e4c5-4335-86bf-2f4b28e0bea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:33.457080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af39689-e4c5-4335-86bf-2f4b28e0bea0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:33.457795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af39689-e4c5-4335-86bf-2f4b28e0bea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:33.458468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af39689-e4c5-4335-86bf-2f4b28e0bea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:33.460081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af39689-e4c5-4335-86bf-2f4b28e0bea0 None admin] RBAC: Adding query filter params (role.id=34ccf81a9f53404e9edefd846827e5b7, scope.project.id=dc51be6e0fb2485a8d71a49a47ea2fb8, user.id=0487c236de2145229f4d07ed5647f6a2) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:33.460830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af39689-e4c5-4335-86bf-2f4b28e0bea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:33.463869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af39689-e4c5-4335-86bf-2f4b28e0bea0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:33.475210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 99/187] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:33 2022] GET /identity/v3/role_assignments?user.id=0487c236de2145229f4d07ed5647f6a2&scope.project.id=dc51be6e0fb2485a8d71a49a47ea2fb8&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:33.564514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 89/188] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:33.569761 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af39689-e4c5-4335-86bf-2f4b28e0bea0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:33.659725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:33.718313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117516f-37e7-4784-b3ad-945ee6685235 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:33.734037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:33.813857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af39689-e4c5-4335-86bf-2f4b28e0bea0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:33.846571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117516f-37e7-4784-b3ad-945ee6685235 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:33.891950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d124d46-3809-4923-b4b3-cc1c0e4ecbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:33.892958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfb585-d03a-466c-ac73-74ad80968308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:33.893598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfb585-d03a-466c-ac73-74ad80968308 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:33.893850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d124d46-3809-4923-b4b3-cc1c0e4ecbff None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:33.894022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfb585-d03a-466c-ac73-74ad80968308 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:33.894171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfb585-d03a-466c-ac73-74ad80968308 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:33.895405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d124d46-3809-4923-b4b3-cc1c0e4ecbff None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:33.896597 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d124d46-3809-4923-b4b3-cc1c0e4ecbff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:33.908406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cfb585-d03a-466c-ac73-74ad80968308 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-17cfb585-d03a-466c-ac73-74ad80968308 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:33.914048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:33.915250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:33.916028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 90/189] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:52:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:33.916708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d124d46-3809-4923-b4b3-cc1c0e4ecbff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:33.916780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfb585-d03a-466c-ac73-74ad80968308 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-2d124d46-3809-4923-b4b3-cc1c0e4ecbff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:33.921735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:33.922711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:33.923813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 100/190] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:33.924476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d124d46-3809-4923-b4b3-cc1c0e4ecbff None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:34.003332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:34.025234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:34.033845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d124d46-3809-4923-b4b3-cc1c0e4ecbff None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:34.036248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfb585-d03a-466c-ac73-74ad80968308 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:34.069036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80c649-d8ca-4418-8482-f4c22d403ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:34.069649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c649-d8ca-4418-8482-f4c22d403ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:34.069930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c649-d8ca-4418-8482-f4c22d403ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:34.070205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c649-d8ca-4418-8482-f4c22d403ec5 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:34.070870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80c649-d8ca-4418-8482-f4c22d403ec5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:34.071201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80c649-d8ca-4418-8482-f4c22d403ec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:34.072170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d6bef-6358-4dd2-9017-78c6dc23b18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:34.072582 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80c649-d8ca-4418-8482-f4c22d403ec5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:34.072797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d6bef-6358-4dd2-9017-78c6dc23b18f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:34.073074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d6bef-6358-4dd2-9017-78c6dc23b18f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:34.073342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d6bef-6358-4dd2-9017-78c6dc23b18f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:34.080859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 101/191] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:34.081676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80c649-d8ca-4418-8482-f4c22d403ec5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:34.084056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d6bef-6358-4dd2-9017-78c6dc23b18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-804d6bef-6358-4dd2-9017-78c6dc23b18f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:34.089073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:34.090065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:34.090850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 91/192] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:52:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:34.092368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d6bef-6358-4dd2-9017-78c6dc23b18f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:34.189601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80c649-d8ca-4418-8482-f4c22d403ec5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:34.198139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d6bef-6358-4dd2-9017-78c6dc23b18f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:34.227204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297cd48-6ac2-4d5f-a229-07e9518786a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:34.227836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297cd48-6ac2-4d5f-a229-07e9518786a7 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:34.228110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297cd48-6ac2-4d5f-a229-07e9518786a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:34.228379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297cd48-6ac2-4d5f-a229-07e9518786a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:34.229050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297cd48-6ac2-4d5f-a229-07e9518786a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:34.229271 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:34.229376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297cd48-6ac2-4d5f-a229-07e9518786a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:34.230736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297cd48-6ac2-4d5f-a229-07e9518786a7 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:34.235790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083fcea-0d5d-40c1-871c-0e793dcd694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:34.236417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083fcea-0d5d-40c1-871c-0e793dcd694d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:34.236692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083fcea-0d5d-40c1-871c-0e793dcd694d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:34.236961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083fcea-0d5d-40c1-871c-0e793dcd694d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:34.238071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083fcea-0d5d-40c1-871c-0e793dcd694d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:34.239540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083fcea-0d5d-40c1-871c-0e793dcd694d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:34.239992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 102/193] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:34.240889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297cd48-6ac2-4d5f-a229-07e9518786a7 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:34.246572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 92/194] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:34.248291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083fcea-0d5d-40c1-871c-0e793dcd694d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:34.261527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:34.351958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297cd48-6ac2-4d5f-a229-07e9518786a7 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:34.356490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083fcea-0d5d-40c1-871c-0e793dcd694d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:34.386906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adf982-71c2-41ce-b039-38939bf05d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:34.387570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adf982-71c2-41ce-b039-38939bf05d11 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:34.388333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adf982-71c2-41ce-b039-38939bf05d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:34.388604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adf982-71c2-41ce-b039-38939bf05d11 None admin] PATH_INFO: `/v3/users/18539fb9031a409bafcebacceaa9b67a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:34.390565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a5f27-8e9b-485f-a81b-2c8b54ae7fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:34.391203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a5f27-8e9b-485f-a81b-2c8b54ae7fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:34.391473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a5f27-8e9b-485f-a81b-2c8b54ae7fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:34.391738 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a5f27-8e9b-485f-a81b-2c8b54ae7fc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:34.392406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a5f27-8e9b-485f-a81b-2c8b54ae7fc8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:34.392702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a5f27-8e9b-485f-a81b-2c8b54ae7fc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:34.394069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a5f27-8e9b-485f-a81b-2c8b54ae7fc8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:34.402591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 93/195] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:52:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:34.402770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6adf982-71c2-41ce-b039-38939bf05d11 None admin] RBAC: Authorizing `identity:get_user(user_id=18539fb9031a409bafcebacceaa9b67a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:34.404294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a5f27-8e9b-485f-a81b-2c8b54ae7fc8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:34.404895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6adf982-71c2-41ce-b039-38939bf05d11 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:34.406674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 103/196] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:34 2022] GET /identity/v3/users/18539fb9031a409bafcebacceaa9b67a => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:34.407333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adf982-71c2-41ce-b039-38939bf05d11 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:34.472514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:34.486979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:34.510520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adf982-71c2-41ce-b039-38939bf05d11 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:34.515272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a5f27-8e9b-485f-a81b-2c8b54ae7fc8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:34.548861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8449611-57ff-40b1-bd83-fc2b1a58f8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:34.549468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8449611-57ff-40b1-bd83-fc2b1a58f8d5 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:34.549739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8449611-57ff-40b1-bd83-fc2b1a58f8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:34.550137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8449611-57ff-40b1-bd83-fc2b1a58f8d5 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:34.550661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8449611-57ff-40b1-bd83-fc2b1a58f8d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:34.550844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e36a7b-e54f-4ddf-acec-21397c38ba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:34.551516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e36a7b-e54f-4ddf-acec-21397c38ba43 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:34.551791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e36a7b-e54f-4ddf-acec-21397c38ba43 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:34.552058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e36a7b-e54f-4ddf-acec-21397c38ba43 None admin] PATH_INFO: `/v3/users/744758ab7db74429aa3497d154d10b77` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:34.552145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8449611-57ff-40b1-bd83-fc2b1a58f8d5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:34.553178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e36a7b-e54f-4ddf-acec-21397c38ba43 None admin] RBAC: Authorizing `identity:get_user(user_id=744758ab7db74429aa3497d154d10b77)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:34.554590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e36a7b-e54f-4ddf-acec-21397c38ba43 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:34.556190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 94/197] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:34 2022] GET /identity/v3/users/744758ab7db74429aa3497d154d10b77 => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:34.561377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e36a7b-e54f-4ddf-acec-21397c38ba43 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:34.574189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 104/198] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:52:34 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:34.587244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8449611-57ff-40b1-bd83-fc2b1a58f8d5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:34.675906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e36a7b-e54f-4ddf-acec-21397c38ba43 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:34.710503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8449611-57ff-40b1-bd83-fc2b1a58f8d5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:34.715422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:34.724821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f33ce2-01aa-48af-aa75-483560dff253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:34.725478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f33ce2-01aa-48af-aa75-483560dff253 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:34.725755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f33ce2-01aa-48af-aa75-483560dff253 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:34.726026 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f33ce2-01aa-48af-aa75-483560dff253 None admin] PATH_INFO: `/v3/users/18539fb9031a409bafcebacceaa9b67a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:34.727270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f33ce2-01aa-48af-aa75-483560dff253 None admin] RBAC: Authorizing `identity:get_user(user_id=18539fb9031a409bafcebacceaa9b67a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:34.728720 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f33ce2-01aa-48af-aa75-483560dff253 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:34.730349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 95/199] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:34 2022] GET /identity/v3/users/18539fb9031a409bafcebacceaa9b67a => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:34.731200 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f33ce2-01aa-48af-aa75-483560dff253 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:34.758469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:34.765845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c487524-527f-4f71-b254-de7ff205eeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:34.767048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c487524-527f-4f71-b254-de7ff205eeff None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:34.767579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c487524-527f-4f71-b254-de7ff205eeff None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:34.767579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c487524-527f-4f71-b254-de7ff205eeff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:34.856058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c487524-527f-4f71-b254-de7ff205eeff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:34.858588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c487524-527f-4f71-b254-de7ff205eeff None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:34.859670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f33ce2-01aa-48af-aa75-483560dff253 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:34.904744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a474a6-4636-4866-b476-48ff151b87e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:34.905638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a474a6-4636-4866-b476-48ff151b87e0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:34.905914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a474a6-4636-4866-b476-48ff151b87e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:34.906184 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a474a6-4636-4866-b476-48ff151b87e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:34.991771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:35.000130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a474a6-4636-4866-b476-48ff151b87e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:35.001526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 105/200] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:35.002169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a474a6-4636-4866-b476-48ff151b87e0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:35.003019 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c487524-527f-4f71-b254-de7ff205eeff None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:35.047042 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:35.120080 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 96/201] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:35.125703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a474a6-4636-4866-b476-48ff151b87e0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:35.147838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c487524-527f-4f71-b254-de7ff205eeff None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:35.193165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53402b72-9b78-4c40-a388-9617c9ac6554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:35.194724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53402b72-9b78-4c40-a388-9617c9ac6554 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:35.195240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53402b72-9b78-4c40-a388-9617c9ac6554 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:35.195995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53402b72-9b78-4c40-a388-9617c9ac6554 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:35.255515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:35.264644 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a474a6-4636-4866-b476-48ff151b87e0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:35.282789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53402b72-9b78-4c40-a388-9617c9ac6554 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:35.284451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53402b72-9b78-4c40-a388-9617c9ac6554 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:35.314623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eff9a6-f8e6-4be4-affb-7a7e5922a8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:35.315856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eff9a6-f8e6-4be4-affb-7a7e5922a8ff None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:35.316671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eff9a6-f8e6-4be4-affb-7a7e5922a8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:35.317259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eff9a6-f8e6-4be4-affb-7a7e5922a8ff None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:35.318929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eff9a6-f8e6-4be4-affb-7a7e5922a8ff None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:35.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:35.322133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eff9a6-f8e6-4be4-affb-7a7e5922a8ff None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:35.324231 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 97/202] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:35 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:35.325229 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eff9a6-f8e6-4be4-affb-7a7e5922a8ff None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:35.427769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 106/203] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:35.429221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53402b72-9b78-4c40-a388-9617c9ac6554 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:35.441980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eff9a6-f8e6-4be4-affb-7a7e5922a8ff None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:35.489582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea454adf-264b-4566-9bcd-3e5116599200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:35.490327 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea454adf-264b-4566-9bcd-3e5116599200 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:35.490703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea454adf-264b-4566-9bcd-3e5116599200 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:35.491102 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea454adf-264b-4566-9bcd-3e5116599200 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:35.492445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea454adf-264b-4566-9bcd-3e5116599200 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:35.494051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea454adf-264b-4566-9bcd-3e5116599200 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:35.499288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 98/204] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:35 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:35.499966 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea454adf-264b-4566-9bcd-3e5116599200 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:35.544108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:35.576073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53402b72-9b78-4c40-a388-9617c9ac6554 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:35.618639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:35.625488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea454adf-264b-4566-9bcd-3e5116599200 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:35.626647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6d336-f455-44c1-b387-95314fc094a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:35.627469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6d336-f455-44c1-b387-95314fc094a1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:35.628107 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6d336-f455-44c1-b387-95314fc094a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:35.628549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6d336-f455-44c1-b387-95314fc094a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:35.630396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6d336-f455-44c1-b387-95314fc094a1 None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542, user.id=18539fb9031a409bafcebacceaa9b67a) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:35.630859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6d336-f455-44c1-b387-95314fc094a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:35.632514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6d336-f455-44c1-b387-95314fc094a1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:35.640386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 107/205] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:35 2022] GET /identity/v3/role_assignments?user.id=18539fb9031a409bafcebacceaa9b67a&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:35.641450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6d336-f455-44c1-b387-95314fc094a1 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:35.703180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6d229-b9a6-4f1d-bb76-2765d69545c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:35.705405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6d229-b9a6-4f1d-bb76-2765d69545c8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:35.707403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6d229-b9a6-4f1d-bb76-2765d69545c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:35.707990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6d229-b9a6-4f1d-bb76-2765d69545c8 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:35.711506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6d229-b9a6-4f1d-bb76-2765d69545c8 None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:35.714496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6d229-b9a6-4f1d-bb76-2765d69545c8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:35.719612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 99/206] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:35 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:35.726345 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6d229-b9a6-4f1d-bb76-2765d69545c8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:35.879664 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:35.881039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6d336-f455-44c1-b387-95314fc094a1 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:35.885794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6d229-b9a6-4f1d-bb76-2765d69545c8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:35.937596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429b647-ca93-41c2-bccc-edecc18829e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:35.939272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429b647-ca93-41c2-bccc-edecc18829e5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:35.939867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429b647-ca93-41c2-bccc-edecc18829e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:35.940506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429b647-ca93-41c2-bccc-edecc18829e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:35.942589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429b647-ca93-41c2-bccc-edecc18829e5 None admin] RBAC: Adding query filter params (role.id=372b98bd7cdc4f7fbd4b812a50380cef, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542, user.id=18539fb9031a409bafcebacceaa9b67a) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:35.943151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429b647-ca93-41c2-bccc-edecc18829e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:35.945512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429b647-ca93-41c2-bccc-edecc18829e5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:35.954706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101a381-79ca-4927-bcac-fe2442dd371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:35.955854 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101a381-79ca-4927-bcac-fe2442dd371f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:35.957446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101a381-79ca-4927-bcac-fe2442dd371f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:35.957921 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101a381-79ca-4927-bcac-fe2442dd371f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:35.959578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101a381-79ca-4927-bcac-fe2442dd371f None admin] RBAC: Adding query filter params (role.id=0ab8ef19a25b4e489c9abb312698599c, scope.project.id=ba7f09585e054b049eec4280fdd8896c, user.id=744758ab7db74429aa3497d154d10b77) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:35.960120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 100/207] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:35 2022] GET /identity/v3/role_assignments?user.id=18539fb9031a409bafcebacceaa9b67a&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=372b98bd7cdc4f7fbd4b812a50380cef => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:35.960670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101a381-79ca-4927-bcac-fe2442dd371f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:35.962374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101a381-79ca-4927-bcac-fe2442dd371f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:35.970236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 108/208] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:35 2022] GET /identity/v3/role_assignments?user.id=744758ab7db74429aa3497d154d10b77&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=0ab8ef19a25b4e489c9abb312698599c => generated 608 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:37.083530 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf87559-70c4-40b6-978d-b801647c9985 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:37.084230 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf87559-70c4-40b6-978d-b801647c9985 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:37.084230 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf87559-70c4-40b6-978d-b801647c9985 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:37.085525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 101/209] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:37.094581 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c537b-d494-4b46-98b6-334a07e0931e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:37.095942 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c537b-d494-4b46-98b6-334a07e0931e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:37.096447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c537b-d494-4b46-98b6-334a07e0931e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:37.105075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97531b-240f-4323-98d3-571ab00465ea None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:37.105356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97531b-240f-4323-98d3-571ab00465ea None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:37.105620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97531b-240f-4323-98d3-571ab00465ea None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:37.107722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 102/210] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:37.112766 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9825-8cc8-4d0b-8d51-5a2867bdc08e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:37.113047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9825-8cc8-4d0b-8d51-5a2867bdc08e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:37.113320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9825-8cc8-4d0b-8d51-5a2867bdc08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:37.120993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-424c537b-d494-4b46-98b6-334a07e0931e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:37.122827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-424c537b-d494-4b46-98b6-334a07e0931e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:37.138605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-37de9825-8cc8-4d0b-8d51-5a2867bdc08e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:37.139888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-37de9825-8cc8-4d0b-8d51-5a2867bdc08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:37.212490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:37.224686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 109/211] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:37.237946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 103/212] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:37.508247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c537b-d494-4b46-98b6-334a07e0931e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:37.510158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de9825-8cc8-4d0b-8d51-5a2867bdc08e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:37.616355 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c537b-d494-4b46-98b6-334a07e0931e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:37.624231 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:37.633491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de9825-8cc8-4d0b-8d51-5a2867bdc08e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:37.654987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe25c1f-12bf-428b-a048-38d55eec4d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:37.655797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe25c1f-12bf-428b-a048-38d55eec4d73 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:37.656083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe25c1f-12bf-428b-a048-38d55eec4d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:37.656352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe25c1f-12bf-428b-a048-38d55eec4d73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:37.666939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe25c1f-12bf-428b-a048-38d55eec4d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-4fe25c1f-12bf-428b-a048-38d55eec4d73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:37.672251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:37.673131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:37.674184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:37.678632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 110/213] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:37.679353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003cdec-d6eb-422d-86b5-1c1fb1b94065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:37.679549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe25c1f-12bf-428b-a048-38d55eec4d73 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:37.680399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003cdec-d6eb-422d-86b5-1c1fb1b94065 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:37.680877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003cdec-d6eb-422d-86b5-1c1fb1b94065 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:37.681337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003cdec-d6eb-422d-86b5-1c1fb1b94065 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:37.682717 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003cdec-d6eb-422d-86b5-1c1fb1b94065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:37.684494 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003cdec-d6eb-422d-86b5-1c1fb1b94065 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:37.691630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 104/214] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:37.692829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003cdec-d6eb-422d-86b5-1c1fb1b94065 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:37.733687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:37.792194 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe25c1f-12bf-428b-a048-38d55eec4d73 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:37.808490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003cdec-d6eb-422d-86b5-1c1fb1b94065 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:37.828255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5dbe8-30a8-4a46-b36b-a78f2edd590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:37.828792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5dbe8-30a8-4a46-b36b-a78f2edd590d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:37.829069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5dbe8-30a8-4a46-b36b-a78f2edd590d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:37.829336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5dbe8-30a8-4a46-b36b-a78f2edd590d None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:37.830014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5dbe8-30a8-4a46-b36b-a78f2edd590d None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:37.830458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5dbe8-30a8-4a46-b36b-a78f2edd590d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:37.831734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5dbe8-30a8-4a46-b36b-a78f2edd590d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:37.840505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 111/215] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:37.847062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5dbe8-30a8-4a46-b36b-a78f2edd590d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:37.850154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be4fd5-f263-4849-be38-16ebb9c4d2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:37.851310 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be4fd5-f263-4849-be38-16ebb9c4d2d9 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:37.851863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be4fd5-f263-4849-be38-16ebb9c4d2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:37.852329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be4fd5-f263-4849-be38-16ebb9c4d2d9 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:37.853260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be4fd5-f263-4849-be38-16ebb9c4d2d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:37.854893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be4fd5-f263-4849-be38-16ebb9c4d2d9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:37.883862 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 105/216] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:52:37 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:37.972260 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5dbe8-30a8-4a46-b36b-a78f2edd590d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:37.991595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:38.018799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18607217-8a8d-4597-a51a-79d5d82451fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:38.019827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18607217-8a8d-4597-a51a-79d5d82451fb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:38.020334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18607217-8a8d-4597-a51a-79d5d82451fb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:38.020801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18607217-8a8d-4597-a51a-79d5d82451fb None admin] PATH_INFO: `/v3/users/fe00075f4d6d453c9a58c46ccc81be6e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:38.038257 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18607217-8a8d-4597-a51a-79d5d82451fb None admin] RBAC: Authorizing `identity:get_user(user_id=fe00075f4d6d453c9a58c46ccc81be6e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:38.040284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18607217-8a8d-4597-a51a-79d5d82451fb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:38.042780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 112/217] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:37 2022] GET /identity/v3/users/fe00075f4d6d453c9a58c46ccc81be6e => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:38.051172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be4fd5-f263-4849-be38-16ebb9c4d2d9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:38.076741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:38.166487 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5da1a3-d247-4723-ac0d-b6b9c73ddf34 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:38.167055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5da1a3-d247-4723-ac0d-b6b9c73ddf34 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:38.167055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5da1a3-d247-4723-ac0d-b6b9c73ddf34 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:38.168540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 113/218] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:38.175250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be4fd5-f263-4849-be38-16ebb9c4d2d9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:38.176935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5f230-dfff-475d-b45e-893d7589a4db None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:38.177294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5f230-dfff-475d-b45e-893d7589a4db None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:38.177567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5f230-dfff-475d-b45e-893d7589a4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:38.206892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2fd5f230-dfff-475d-b45e-893d7589a4db None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:38.208490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2fd5f230-dfff-475d-b45e-893d7589a4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:38.225121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d77892-20b4-44ba-a2de-c593e2b4807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:38.225762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d77892-20b4-44ba-a2de-c593e2b4807f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:38.226039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d77892-20b4-44ba-a2de-c593e2b4807f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:38.226311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d77892-20b4-44ba-a2de-c593e2b4807f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:38.303725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 114/219] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:38.319408 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d77892-20b4-44ba-a2de-c593e2b4807f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:38.320892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d77892-20b4-44ba-a2de-c593e2b4807f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:38.341772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:38.355508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae18743-3dd9-46b4-814c-df39923a0cdb None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:38.355862 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae18743-3dd9-46b4-814c-df39923a0cdb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:38.356072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae18743-3dd9-46b4-814c-df39923a0cdb None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:38.357654 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 115/220] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:38.363798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197f6c2-5e8e-4220-a45a-67b36c3d5d4a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:38.364084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197f6c2-5e8e-4220-a45a-67b36c3d5d4a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:38.364362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197f6c2-5e8e-4220-a45a-67b36c3d5d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:38.392999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7197f6c2-5e8e-4220-a45a-67b36c3d5d4a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:38.394081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7197f6c2-5e8e-4220-a45a-67b36c3d5d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:38.447243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 106/221] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:38.457370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d77892-20b4-44ba-a2de-c593e2b4807f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:38.488813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 116/222] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:38.575909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d77892-20b4-44ba-a2de-c593e2b4807f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:38.594680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:38.610811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197f6c2-5e8e-4220-a45a-67b36c3d5d4a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:38.615014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a94073-ccd8-4c7e-8d57-ceca29928be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:38.615665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a94073-ccd8-4c7e-8d57-ceca29928be3 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:38.615946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a94073-ccd8-4c7e-8d57-ceca29928be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:38.616217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a94073-ccd8-4c7e-8d57-ceca29928be3 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:38.618404 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a94073-ccd8-4c7e-8d57-ceca29928be3 None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:38.620265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a94073-ccd8-4c7e-8d57-ceca29928be3 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:38.622823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 107/223] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:38 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:38.631846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a94073-ccd8-4c7e-8d57-ceca29928be3 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:38.665981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:38.809014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a94073-ccd8-4c7e-8d57-ceca29928be3 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:38.821797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197f6c2-5e8e-4220-a45a-67b36c3d5d4a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:38.868320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c752aa6-3571-46c8-b58b-7da7d6d67fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:38.868915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c752aa6-3571-46c8-b58b-7da7d6d67fdc None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:38.869297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c752aa6-3571-46c8-b58b-7da7d6d67fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:38.869587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c752aa6-3571-46c8-b58b-7da7d6d67fdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:38.878017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59470b-5b40-4385-9fd1-1bc5578eb7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:38.879604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59470b-5b40-4385-9fd1-1bc5578eb7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:38.880082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59470b-5b40-4385-9fd1-1bc5578eb7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:38.880545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59470b-5b40-4385-9fd1-1bc5578eb7f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:38.881597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59470b-5b40-4385-9fd1-1bc5578eb7f6 None admin] RBAC: Adding query filter params (role.id=34ccf81a9f53404e9edefd846827e5b7, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542, user.id=fe00075f4d6d453c9a58c46ccc81be6e) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:38.882196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59470b-5b40-4385-9fd1-1bc5578eb7f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:38.882402 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c752aa6-3571-46c8-b58b-7da7d6d67fdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:38.884080 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59470b-5b40-4385-9fd1-1bc5578eb7f6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-6c752aa6-3571-46c8-b58b-7da7d6d67fdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:38.887648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:38.888538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:38.889311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 117/224] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:52:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:38.890515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c752aa6-3571-46c8-b58b-7da7d6d67fdc None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:38.894169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 108/225] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:38 2022] GET /identity/v3/role_assignments?user.id=fe00075f4d6d453c9a58c46ccc81be6e&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:38.896203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59470b-5b40-4385-9fd1-1bc5578eb7f6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:38.957876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:39.021157 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59470b-5b40-4385-9fd1-1bc5578eb7f6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:39.064365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6185af1-e479-4050-ac21-296758f0a6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:39.065394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6185af1-e479-4050-ac21-296758f0a6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:39.065866 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6185af1-e479-4050-ac21-296758f0a6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:39.067133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6185af1-e479-4050-ac21-296758f0a6a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:39.068137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6185af1-e479-4050-ac21-296758f0a6a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:39.068646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6185af1-e479-4050-ac21-296758f0a6a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:39.070311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6185af1-e479-4050-ac21-296758f0a6a3 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:39.081205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 109/226] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:39.086657 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6185af1-e479-4050-ac21-296758f0a6a3 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:39.130393 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:39.170834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c752aa6-3571-46c8-b58b-7da7d6d67fdc None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:39.250300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6185af1-e479-4050-ac21-296758f0a6a3 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:39.254739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60613751-f980-4ab2-8179-48b5c8538070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:39.255603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60613751-f980-4ab2-8179-48b5c8538070 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:39.255984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60613751-f980-4ab2-8179-48b5c8538070 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:39.256362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60613751-f980-4ab2-8179-48b5c8538070 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:39.274534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60613751-f980-4ab2-8179-48b5c8538070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:39.279307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-60613751-f980-4ab2-8179-48b5c8538070 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:39.280671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:39.281588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:39.282373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:39.283605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 118/227] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:52:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:39.289966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60613751-f980-4ab2-8179-48b5c8538070 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:39.298897 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016cdaeb-7ade-46c3-9b9e-2e8ea564adda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:39.300449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cdaeb-7ade-46c3-9b9e-2e8ea564adda None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:39.300912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cdaeb-7ade-46c3-9b9e-2e8ea564adda None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:39.301359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cdaeb-7ade-46c3-9b9e-2e8ea564adda None admin] PATH_INFO: `/v3/users/18539fb9031a409bafcebacceaa9b67a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:39.303288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016cdaeb-7ade-46c3-9b9e-2e8ea564adda None admin] RBAC: Authorizing `identity:get_user(user_id=18539fb9031a409bafcebacceaa9b67a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:39.305623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016cdaeb-7ade-46c3-9b9e-2e8ea564adda None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:39.308260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 110/228] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:39 2022] GET /identity/v3/users/18539fb9031a409bafcebacceaa9b67a => generated 283 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:39.315457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016cdaeb-7ade-46c3-9b9e-2e8ea564adda None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:39.419763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016cdaeb-7ade-46c3-9b9e-2e8ea564adda None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:39.423748 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60613751-f980-4ab2-8179-48b5c8538070 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:39.463324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b86a1-132c-4deb-a794-dd831dd5c76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:39.464270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b86a1-132c-4deb-a794-dd831dd5c76e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:39.465669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b86a1-132c-4deb-a794-dd831dd5c76e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:39.466244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b86a1-132c-4deb-a794-dd831dd5c76e None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:39.473592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92965603-efc1-4eea-a124-72b6ff1b37dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:39.473592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92965603-efc1-4eea-a124-72b6ff1b37dd None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:39.473592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92965603-efc1-4eea-a124-72b6ff1b37dd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:39.473592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92965603-efc1-4eea-a124-72b6ff1b37dd None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:39.473930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92965603-efc1-4eea-a124-72b6ff1b37dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:39.473930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92965603-efc1-4eea-a124-72b6ff1b37dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:39.473930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92965603-efc1-4eea-a124-72b6ff1b37dd None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:39.475194 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b86a1-132c-4deb-a794-dd831dd5c76e None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:39.476712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b86a1-132c-4deb-a794-dd831dd5c76e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:39.478671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 111/229] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:39 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:39.486323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b86a1-132c-4deb-a794-dd831dd5c76e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:39.489262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 119/230] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:52:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:39.497752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92965603-efc1-4eea-a124-72b6ff1b37dd None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:39.531333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:39.615344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:39.708164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92965603-efc1-4eea-a124-72b6ff1b37dd None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:39.717827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b86a1-132c-4deb-a794-dd831dd5c76e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:39.764553 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aca066-543b-4266-8155-289ac0ed1167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:39.765328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aca066-543b-4266-8155-289ac0ed1167 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:39.765608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aca066-543b-4266-8155-289ac0ed1167 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:39.765881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aca066-543b-4266-8155-289ac0ed1167 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/users/18539fb9031a409bafcebacceaa9b67a/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:39.768009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aca066-543b-4266-8155-289ac0ed1167 None admin] RBAC: Authorizing `identity:create_grant(project_id=c24becb4b27f43ec9eeb7b0ed1066542, user_id=18539fb9031a409bafcebacceaa9b67a, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:39.769751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aca066-543b-4266-8155-289ac0ed1167 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:39.781869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b7d1c-654c-4345-bcb0-52a00048a691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:39.782519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b7d1c-654c-4345-bcb0-52a00048a691 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:39.782799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b7d1c-654c-4345-bcb0-52a00048a691 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:39.784307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b7d1c-654c-4345-bcb0-52a00048a691 None admin] PATH_INFO: `/v3/users/18539fb9031a409bafcebacceaa9b67a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:39.786255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b7d1c-654c-4345-bcb0-52a00048a691 None admin] RBAC: Authorizing `identity:get_user(user_id=18539fb9031a409bafcebacceaa9b67a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:39.790259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 112/231] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:52:39 2022] PUT /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/users/18539fb9031a409bafcebacceaa9b67a/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:52:39.790891 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b7d1c-654c-4345-bcb0-52a00048a691 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:39.792688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 120/232] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:39 2022] GET /identity/v3/users/18539fb9031a409bafcebacceaa9b67a => generated 283 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:39.798861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aca066-543b-4266-8155-289ac0ed1167 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:39.940948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aca066-543b-4266-8155-289ac0ed1167 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:39.952340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:39.988181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03458f18-32b9-4c54-9c95-afaf9033f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:39.988742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458f18-32b9-4c54-9c95-afaf9033f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:39.989041 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458f18-32b9-4c54-9c95-afaf9033f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:39.989374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458f18-32b9-4c54-9c95-afaf9033f7c5 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:39.990655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03458f18-32b9-4c54-9c95-afaf9033f7c5 None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:39.992307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03458f18-32b9-4c54-9c95-afaf9033f7c5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:39.994182 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 113/233] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:39 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:40.004191 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b7d1c-654c-4345-bcb0-52a00048a691 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:40.129648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:40.151092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b7d1c-654c-4345-bcb0-52a00048a691 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:40.194367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e5190-612c-4198-96f5-9d87b5827eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:40.195908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e5190-612c-4198-96f5-9d87b5827eaf None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:40.196394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e5190-612c-4198-96f5-9d87b5827eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:40.197098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e5190-612c-4198-96f5-9d87b5827eaf None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/users/18539fb9031a409bafcebacceaa9b67a/roles/372b98bd7cdc4f7fbd4b812a50380cef` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:40.200722 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e5190-612c-4198-96f5-9d87b5827eaf None admin] RBAC: Authorizing `identity:create_grant(project_id=c24becb4b27f43ec9eeb7b0ed1066542, user_id=18539fb9031a409bafcebacceaa9b67a, role_id=372b98bd7cdc4f7fbd4b812a50380cef)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:40.202543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e5190-612c-4198-96f5-9d87b5827eaf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:40.223925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 121/234] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:52:40 2022] PUT /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/users/18539fb9031a409bafcebacceaa9b67a/roles/372b98bd7cdc4f7fbd4b812a50380cef => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:52:40.808201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb1bab-dae3-48d5-94ea-73cfdb961471 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:40.808715 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb1bab-dae3-48d5-94ea-73cfdb961471 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:40.808715 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb1bab-dae3-48d5-94ea-73cfdb961471 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:40.810781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 114/235] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:40.816831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5b407-93e3-492d-933b-08fb21ea0505 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:40.817435 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5b407-93e3-492d-933b-08fb21ea0505 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:40.817907 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5b407-93e3-492d-933b-08fb21ea0505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:40.844695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2de5b407-93e3-492d-933b-08fb21ea0505 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:40.846317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2de5b407-93e3-492d-933b-08fb21ea0505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:40.945661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 122/236] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:41.001860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b5d95-7b97-4caf-a9cb-076530c32bd3 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:41.002489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b5d95-7b97-4caf-a9cb-076530c32bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:41.003022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b5d95-7b97-4caf-a9cb-076530c32bd3 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:41.004914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 115/237] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:41.011916 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438a81e-57be-4403-9552-b1ae3468e912 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:41.012206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438a81e-57be-4403-9552-b1ae3468e912 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:41.012481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438a81e-57be-4403-9552-b1ae3468e912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:41.040117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c438a81e-57be-4403-9552-b1ae3468e912 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:41.041241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c438a81e-57be-4403-9552-b1ae3468e912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:41.124592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:41.131307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 123/238] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:41.327808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b5d95-7b97-4caf-a9cb-076530c32bd3 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:41.423600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438a81e-57be-4403-9552-b1ae3468e912 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:41.575361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438a81e-57be-4403-9552-b1ae3468e912 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:41.583751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b5d95-7b97-4caf-a9cb-076530c32bd3 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:41.612619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b511409-e5a7-4309-b1e9-d9e01dc4a5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:41.613264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b511409-e5a7-4309-b1e9-d9e01dc4a5de None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:41.613544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b511409-e5a7-4309-b1e9-d9e01dc4a5de None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:41.613816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b511409-e5a7-4309-b1e9-d9e01dc4a5de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:41.614984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b511409-e5a7-4309-b1e9-d9e01dc4a5de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:41.616433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b511409-e5a7-4309-b1e9-d9e01dc4a5de None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:41.618039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 124/239] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:41.624655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b511409-e5a7-4309-b1e9-d9e01dc4a5de None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:41.667133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4544-bacf-4e43-a52c-78255acac7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:41.667885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4544-bacf-4e43-a52c-78255acac7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:41.668169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4544-bacf-4e43-a52c-78255acac7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:41.668443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4544-bacf-4e43-a52c-78255acac7a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:41.691646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d4544-bacf-4e43-a52c-78255acac7a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-844d4544-bacf-4e43-a52c-78255acac7a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:41.697449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:41.698349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:41.699205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:41.699817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:41.703359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 116/240] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:52:41.716815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4544-bacf-4e43-a52c-78255acac7a8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:41.754780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b511409-e5a7-4309-b1e9-d9e01dc4a5de None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:41.791873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc79095-6893-40cf-9d9c-4c0f4ff1778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:41.792486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc79095-6893-40cf-9d9c-4c0f4ff1778f None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:41.792765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc79095-6893-40cf-9d9c-4c0f4ff1778f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:41.793037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc79095-6893-40cf-9d9c-4c0f4ff1778f None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:41.793693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc79095-6893-40cf-9d9c-4c0f4ff1778f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:41.795714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc79095-6893-40cf-9d9c-4c0f4ff1778f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:41.802868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:41.834679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 125/241] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:52:41 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:41.927871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4544-bacf-4e43-a52c-78255acac7a8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:41.978222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f05a1-d88e-49df-b084-1143a3b66e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:41.978866 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f05a1-d88e-49df-b084-1143a3b66e0a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:41.979232 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f05a1-d88e-49df-b084-1143a3b66e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:41.979512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f05a1-d88e-49df-b084-1143a3b66e0a None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:41.980373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f05a1-d88e-49df-b084-1143a3b66e0a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:41.980689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f05a1-d88e-49df-b084-1143a3b66e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:41.982692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f05a1-d88e-49df-b084-1143a3b66e0a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:41.993145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 117/242] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:41.999534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc79095-6893-40cf-9d9c-4c0f4ff1778f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:42.107164 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7c0b1-f45c-4d56-9de2-8020f45af578 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:42.107660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7c0b1-f45c-4d56-9de2-8020f45af578 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:42.108103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7c0b1-f45c-4d56-9de2-8020f45af578 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:42.110506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 118/243] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:42.115178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2fbfc-7edf-4843-b9f9-dc0251c8a879 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:42.115667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2fbfc-7edf-4843-b9f9-dc0251c8a879 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:42.116082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2fbfc-7edf-4843-b9f9-dc0251c8a879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:42.141246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc79095-6893-40cf-9d9c-4c0f4ff1778f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:42.144950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-22a2fbfc-7edf-4843-b9f9-dc0251c8a879 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:42.146162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-22a2fbfc-7edf-4843-b9f9-dc0251c8a879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:42.172233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:42.192942 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6922b-a451-405a-9315-d55571dfb2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:42.193611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6922b-a451-405a-9315-d55571dfb2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:42.194735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6922b-a451-405a-9315-d55571dfb2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:42.194735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6922b-a451-405a-9315-d55571dfb2b9 None admin] PATH_INFO: `/v3/users/7d6a80d272564afc95859a3bc9c79c0e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:42.217393 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6922b-a451-405a-9315-d55571dfb2b9 None admin] RBAC: Authorizing `identity:get_user(user_id=7d6a80d272564afc95859a3bc9c79c0e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:42.220781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6922b-a451-405a-9315-d55571dfb2b9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:42.230640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 126/244] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:41 2022] GET /identity/v3/users/7d6a80d272564afc95859a3bc9c79c0e => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:42.239047 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6922b-a451-405a-9315-d55571dfb2b9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:42.276803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 119/245] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:42.361238 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6922b-a451-405a-9315-d55571dfb2b9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:42.404180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adcc74-1190-43db-9a24-84f8f8c72d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:42.404823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adcc74-1190-43db-9a24-84f8f8c72d5e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:42.405092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adcc74-1190-43db-9a24-84f8f8c72d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:42.405364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adcc74-1190-43db-9a24-84f8f8c72d5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:42.463398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:42.483238 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adcc74-1190-43db-9a24-84f8f8c72d5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:42.484723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adcc74-1190-43db-9a24-84f8f8c72d5e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:42.596549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 127/246] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 16:52:42.604860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2fbfc-7edf-4843-b9f9-dc0251c8a879 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:42.612053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adcc74-1190-43db-9a24-84f8f8c72d5e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:42.655630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:42.718877 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:42.740341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adcc74-1190-43db-9a24-84f8f8c72d5e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:42.786520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f750a-99b3-4407-8a85-085c2224895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:42.787243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f750a-99b3-4407-8a85-085c2224895e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:42.787531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f750a-99b3-4407-8a85-085c2224895e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:42.787803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f750a-99b3-4407-8a85-085c2224895e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:42.793466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2fbfc-7edf-4843-b9f9-dc0251c8a879 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:42.800834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f750a-99b3-4407-8a85-085c2224895e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-797f750a-99b3-4407-8a85-085c2224895e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:42.805852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:42.806783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:42.807798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 128/247] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:52:42.815327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7630d-2f21-4ae4-af92-2b4072478748 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:42.815676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7630d-2f21-4ae4-af92-2b4072478748 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:42.815960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7630d-2f21-4ae4-af92-2b4072478748 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:42.817423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 129/248] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:42.821724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7630d-2f21-4ae4-af92-2b4072478748 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:42.842677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfdfa1-3764-4c77-88d3-f8eb4b067232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:42.843731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfdfa1-3764-4c77-88d3-f8eb4b067232 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:42.844206 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfdfa1-3764-4c77-88d3-f8eb4b067232 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:42.844669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfdfa1-3764-4c77-88d3-f8eb4b067232 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:42.846631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfdfa1-3764-4c77-88d3-f8eb4b067232 None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:42.848828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfdfa1-3764-4c77-88d3-f8eb4b067232 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:42.851969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 120/249] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:42 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:42.854102 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f76fc-b723-47c8-b516-abb905fb933f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:42.854568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f76fc-b723-47c8-b516-abb905fb933f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:42.855282 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f76fc-b723-47c8-b516-abb905fb933f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:42.879777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-312f76fc-b723-47c8-b516-abb905fb933f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:42.881269 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-312f76fc-b723-47c8-b516-abb905fb933f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:42.956253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7630d-2f21-4ae4-af92-2b4072478748 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:43.024690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a8a06-1cf1-46c7-82de-5308effe19ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:43.025312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a8a06-1cf1-46c7-82de-5308effe19ee None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:43.025598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a8a06-1cf1-46c7-82de-5308effe19ee None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:43.025870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a8a06-1cf1-46c7-82de-5308effe19ee None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:43.026543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a8a06-1cf1-46c7-82de-5308effe19ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:43.026849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a8a06-1cf1-46c7-82de-5308effe19ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:43.028325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a8a06-1cf1-46c7-82de-5308effe19ee None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:43.034782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 121/250] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:43.035727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f76fc-b723-47c8-b516-abb905fb933f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:43.052594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 130/251] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:43.061168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a8a06-1cf1-46c7-82de-5308effe19ee None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:43.067756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:43.102377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:43.228731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f76fc-b723-47c8-b516-abb905fb933f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:43.258521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a8a06-1cf1-46c7-82de-5308effe19ee None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:43.292866 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee0f9d-b146-4c2a-9fef-6d14736c4969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:43.294383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0f9d-b146-4c2a-9fef-6d14736c4969 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:43.294860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0f9d-b146-4c2a-9fef-6d14736c4969 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:43.295376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0f9d-b146-4c2a-9fef-6d14736c4969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:43.296296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee0f9d-b146-4c2a-9fef-6d14736c4969 None admin] RBAC: Adding query filter params (role.id=0ab8ef19a25b4e489c9abb312698599c, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542, user.id=7d6a80d272564afc95859a3bc9c79c0e) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:43.302896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee0f9d-b146-4c2a-9fef-6d14736c4969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:43.302896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee0f9d-b146-4c2a-9fef-6d14736c4969 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:43.307512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be09377-7459-4417-8ddf-cb30dfbc5941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:43.308240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be09377-7459-4417-8ddf-cb30dfbc5941 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:43.308527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be09377-7459-4417-8ddf-cb30dfbc5941 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:43.308800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be09377-7459-4417-8ddf-cb30dfbc5941 None admin] PATH_INFO: `/v3/users/fe00075f4d6d453c9a58c46ccc81be6e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:43.311048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be09377-7459-4417-8ddf-cb30dfbc5941 None admin] RBAC: Authorizing `identity:get_user(user_id=fe00075f4d6d453c9a58c46ccc81be6e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:43.312512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be09377-7459-4417-8ddf-cb30dfbc5941 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:43.315289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 131/252] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:43 2022] GET /identity/v3/users/fe00075f4d6d453c9a58c46ccc81be6e => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:43.316470 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6941072-e058-4e76-a3f9-f797a480753e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:43.316750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6941072-e058-4e76-a3f9-f797a480753e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:43.317027 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6941072-e058-4e76-a3f9-f797a480753e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:43.324306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 122/253] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:43 2022] GET /identity/v3/role_assignments?user.id=7d6a80d272564afc95859a3bc9c79c0e&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=0ab8ef19a25b4e489c9abb312698599c => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:43.325685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee0f9d-b146-4c2a-9fef-6d14736c4969 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:43.329722 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 132/254] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:43 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:43.332099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8e362-487c-4501-a7a1-151159ae61e3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:43.334210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8e362-487c-4501-a7a1-151159ae61e3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:43.334507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8e362-487c-4501-a7a1-151159ae61e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:43.372353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2aa8e362-487c-4501-a7a1-151159ae61e3 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:43.374759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2aa8e362-487c-4501-a7a1-151159ae61e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:43.495080 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:43.502989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee0f9d-b146-4c2a-9fef-6d14736c4969 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:43.520659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 133/255] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:43.521555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8e362-487c-4501-a7a1-151159ae61e3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:43.524408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:43.568786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69c67e-6ade-494b-9f68-f84a018aef13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:43.569484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69c67e-6ade-494b-9f68-f84a018aef13 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:43.569871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69c67e-6ade-494b-9f68-f84a018aef13 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:43.570152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69c67e-6ade-494b-9f68-f84a018aef13 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:43.573295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69c67e-6ade-494b-9f68-f84a018aef13 None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:43.574848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69c67e-6ade-494b-9f68-f84a018aef13 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:43.577237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 123/256] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:43 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:43.583113 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69c67e-6ade-494b-9f68-f84a018aef13 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:43.678087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8e362-487c-4501-a7a1-151159ae61e3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:43.711163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69c67e-6ade-494b-9f68-f84a018aef13 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:43.749852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8c75d-bde7-43d2-a7f0-e7db557d27b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:43.750430 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8c75d-bde7-43d2-a7f0-e7db557d27b4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:43.750704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8c75d-bde7-43d2-a7f0-e7db557d27b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:43.751006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8c75d-bde7-43d2-a7f0-e7db557d27b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:43.751999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49010022-6db8-48e6-ac2c-f8252d08ffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:43.752754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49010022-6db8-48e6-ac2c-f8252d08ffb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:43.753200 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49010022-6db8-48e6-ac2c-f8252d08ffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:43.753550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49010022-6db8-48e6-ac2c-f8252d08ffb5 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/users/fe00075f4d6d453c9a58c46ccc81be6e/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:43.755739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49010022-6db8-48e6-ac2c-f8252d08ffb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c24becb4b27f43ec9eeb7b0ed1066542, user_id=fe00075f4d6d453c9a58c46ccc81be6e, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:43.757305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49010022-6db8-48e6-ac2c-f8252d08ffb5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:43.764917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8c75d-bde7-43d2-a7f0-e7db557d27b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-79f8c75d-bde7-43d2-a7f0-e7db557d27b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:43.770938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:43.771991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:43.772811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 134/257] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:52:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:43.784186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8c75d-bde7-43d2-a7f0-e7db557d27b4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:43.784694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 124/258] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:52:43 2022] PUT /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/users/fe00075f4d6d453c9a58c46ccc81be6e/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 16:52:43.887375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:43.950552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8c75d-bde7-43d2-a7f0-e7db557d27b4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:43.988445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf1246-dc1f-4920-b6dd-494aef98fc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:43.989064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf1246-dc1f-4920-b6dd-494aef98fc7f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:43.989337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf1246-dc1f-4920-b6dd-494aef98fc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:43.989605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf1246-dc1f-4920-b6dd-494aef98fc7f None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:43.990270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf1246-dc1f-4920-b6dd-494aef98fc7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:43.990567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf1246-dc1f-4920-b6dd-494aef98fc7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:43.992028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf1246-dc1f-4920-b6dd-494aef98fc7f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:44.001711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 135/259] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:44.008057 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49010022-6db8-48e6-ac2c-f8252d08ffb5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:44.047340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:44.144066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf1246-dc1f-4920-b6dd-494aef98fc7f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:44.175508 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49010022-6db8-48e6-ac2c-f8252d08ffb5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:44.233542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89baa7-4d04-4c54-ae1c-9e951562b3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:44.234413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89baa7-4d04-4c54-ae1c-9e951562b3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:44.234767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89baa7-4d04-4c54-ae1c-9e951562b3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:44.235869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89baa7-4d04-4c54-ae1c-9e951562b3f6 None admin] PATH_INFO: `/v3/users/18539fb9031a409bafcebacceaa9b67a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:44.238754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f89baa7-4d04-4c54-ae1c-9e951562b3f6 None admin] RBAC: Authorizing `identity:get_user(user_id=18539fb9031a409bafcebacceaa9b67a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:44.240609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f89baa7-4d04-4c54-ae1c-9e951562b3f6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:44.242576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 125/260] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:44 2022] GET /identity/v3/users/18539fb9031a409bafcebacceaa9b67a => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:44.251323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89baa7-4d04-4c54-ae1c-9e951562b3f6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:44.347048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf1246-dc1f-4920-b6dd-494aef98fc7f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:44.391048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:44.414721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93154dd-346a-4630-a794-2dd15352bb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:44.415680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93154dd-346a-4630-a794-2dd15352bb50 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:44.416343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93154dd-346a-4630-a794-2dd15352bb50 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:44.417869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93154dd-346a-4630-a794-2dd15352bb50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:44.432021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:44.442406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93154dd-346a-4630-a794-2dd15352bb50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b93154dd-346a-4630-a794-2dd15352bb50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:44.448080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:44.448993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:44.449780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:44.455601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 136/261] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:52:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:52:44.459783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93154dd-346a-4630-a794-2dd15352bb50 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:44.466932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89baa7-4d04-4c54-ae1c-9e951562b3f6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:44.532799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721ee63-9c32-4508-9250-3c91fabe91d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:44.533864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721ee63-9c32-4508-9250-3c91fabe91d3 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:44.535281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721ee63-9c32-4508-9250-3c91fabe91d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:44.536593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721ee63-9c32-4508-9250-3c91fabe91d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:44.610470 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93154dd-346a-4630-a794-2dd15352bb50 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:44.663674 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721ee63-9c32-4508-9250-3c91fabe91d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:44.666377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721ee63-9c32-4508-9250-3c91fabe91d3 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:44.683309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8caf3-c1ed-41db-8cb8-ea18c182abdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:44.684316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8caf3-c1ed-41db-8cb8-ea18c182abdf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:44.684867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8caf3-c1ed-41db-8cb8-ea18c182abdf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:44.685339 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8caf3-c1ed-41db-8cb8-ea18c182abdf None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:44.686280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8caf3-c1ed-41db-8cb8-ea18c182abdf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:44.686825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8caf3-c1ed-41db-8cb8-ea18c182abdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:44.688637 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8caf3-c1ed-41db-8cb8-ea18c182abdf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:44.704854 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 137/262] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:52:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:44.710166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8caf3-c1ed-41db-8cb8-ea18c182abdf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:44.787338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:44.795659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:44.824640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 126/263] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:44.835112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721ee63-9c32-4508-9250-3c91fabe91d3 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:44.883409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8caf3-c1ed-41db-8cb8-ea18c182abdf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:44.943261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771052a2-b468-4fe9-b966-c4ca25d49303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:44.943906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771052a2-b468-4fe9-b966-c4ca25d49303 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:44.944192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771052a2-b468-4fe9-b966-c4ca25d49303 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:44.944573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771052a2-b468-4fe9-b966-c4ca25d49303 None admin] PATH_INFO: `/v3/users/18539fb9031a409bafcebacceaa9b67a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:44.945818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771052a2-b468-4fe9-b966-c4ca25d49303 None admin] RBAC: Authorizing `identity:get_user(user_id=18539fb9031a409bafcebacceaa9b67a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:44.955557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771052a2-b468-4fe9-b966-c4ca25d49303 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:44.957851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 138/264] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:44 2022] GET /identity/v3/users/18539fb9031a409bafcebacceaa9b67a => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:44.963429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771052a2-b468-4fe9-b966-c4ca25d49303 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:44.995193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721ee63-9c32-4508-9250-3c91fabe91d3 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:45.042880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838734d-f991-42b2-ac0c-42f8f3af47bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:45.046726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838734d-f991-42b2-ac0c-42f8f3af47bb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:45.047041 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838734d-f991-42b2-ac0c-42f8f3af47bb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:45.047321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838734d-f991-42b2-ac0c-42f8f3af47bb None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:45.048987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838734d-f991-42b2-ac0c-42f8f3af47bb None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:45.050474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838734d-f991-42b2-ac0c-42f8f3af47bb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:45.054994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 127/265] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:44 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:45.060045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838734d-f991-42b2-ac0c-42f8f3af47bb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:45.174937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771052a2-b468-4fe9-b966-c4ca25d49303 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:45.181368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:45.208637 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:45.258999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26e1b3-c690-42f6-a0c3-b1f296b36402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:45.259658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26e1b3-c690-42f6-a0c3-b1f296b36402 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:45.259931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26e1b3-c690-42f6-a0c3-b1f296b36402 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:45.260201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26e1b3-c690-42f6-a0c3-b1f296b36402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:45.279987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838734d-f991-42b2-ac0c-42f8f3af47bb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:45.352701 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba42b4-e48e-4be5-b3a0-40a6d9811264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:45.354014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba42b4-e48e-4be5-b3a0-40a6d9811264 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:45.354736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba42b4-e48e-4be5-b3a0-40a6d9811264 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:45.355274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba42b4-e48e-4be5-b3a0-40a6d9811264 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:45.356420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba42b4-e48e-4be5-b3a0-40a6d9811264 None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542, user.id=18539fb9031a409bafcebacceaa9b67a) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:45.356927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba42b4-e48e-4be5-b3a0-40a6d9811264 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:45.359160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba42b4-e48e-4be5-b3a0-40a6d9811264 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:45.368172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 128/266] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:45 2022] GET /identity/v3/role_assignments?user.id=18539fb9031a409bafcebacceaa9b67a&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:45.374008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aafd73-a9d4-4e60-9a2d-7106e00530dd None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:45.374296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aafd73-a9d4-4e60-9a2d-7106e00530dd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:45.374546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aafd73-a9d4-4e60-9a2d-7106e00530dd None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:45.377987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 129/267] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:45.384625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8170a8-1946-4657-892a-c97d39b43a4a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:45.384944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8170a8-1946-4657-892a-c97d39b43a4a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:45.385162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8170a8-1946-4657-892a-c97d39b43a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:45.393446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26e1b3-c690-42f6-a0c3-b1f296b36402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:45.394940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26e1b3-c690-42f6-a0c3-b1f296b36402 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:45.420523 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8e8170a8-1946-4657-892a-c97d39b43a4a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:45.421980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8e8170a8-1946-4657-892a-c97d39b43a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:45.517019 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 139/268] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 16:52:45.528365 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26e1b3-c690-42f6-a0c3-b1f296b36402 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:45.564003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:45.580370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 130/269] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:45.711976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26e1b3-c690-42f6-a0c3-b1f296b36402 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:45.761973 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ab5cc-2389-46d0-9dbd-47d1430da4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:45.762675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ab5cc-2389-46d0-9dbd-47d1430da4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:45.762973 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ab5cc-2389-46d0-9dbd-47d1430da4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:45.763262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ab5cc-2389-46d0-9dbd-47d1430da4b4 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:45.764550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ab5cc-2389-46d0-9dbd-47d1430da4b4 None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:45.766460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ab5cc-2389-46d0-9dbd-47d1430da4b4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:45.767905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 140/270] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:45 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:45.768454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090b04-ae06-4b3d-9cdd-64798d1dacdd None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:45.768707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090b04-ae06-4b3d-9cdd-64798d1dacdd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:45.768982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090b04-ae06-4b3d-9cdd-64798d1dacdd None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:45.770468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 131/271] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:52:45.773309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ab5cc-2389-46d0-9dbd-47d1430da4b4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:45.775701 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09045bf7-3717-413e-a61c-c97069025c2b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:45.776119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09045bf7-3717-413e-a61c-c97069025c2b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:45.776400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09045bf7-3717-413e-a61c-c97069025c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:45.800882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-09045bf7-3717-413e-a61c-c97069025c2b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:45.802091 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-09045bf7-3717-413e-a61c-c97069025c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:45.888827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:45.911177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ab5cc-2389-46d0-9dbd-47d1430da4b4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:45.918428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:45.942630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 132/272] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:45.960491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8d1b8-8623-48b4-9ac0-499280954ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:45.961121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d1b8-8623-48b4-9ac0-499280954ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:45.961401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d1b8-8623-48b4-9ac0-499280954ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:45.961678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d1b8-8623-48b4-9ac0-499280954ba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:45.962412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8d1b8-8623-48b4-9ac0-499280954ba8 None admin] RBAC: Adding query filter params (role.id=372b98bd7cdc4f7fbd4b812a50380cef, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542, user.id=18539fb9031a409bafcebacceaa9b67a) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:45.962719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8d1b8-8623-48b4-9ac0-499280954ba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:45.964200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8d1b8-8623-48b4-9ac0-499280954ba8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:45.971505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 141/273] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:45 2022] GET /identity/v3/role_assignments?user.id=18539fb9031a409bafcebacceaa9b67a&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=372b98bd7cdc4f7fbd4b812a50380cef => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:46.070925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09045bf7-3717-413e-a61c-c97069025c2b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:46.213476 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09045bf7-3717-413e-a61c-c97069025c2b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:46.244209 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8d1b8-8623-48b4-9ac0-499280954ba8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:46.260692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d27aa-d1f9-4ded-ad99-e43a0197742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:46.261643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d27aa-d1f9-4ded-ad99-e43a0197742a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:46.261926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d27aa-d1f9-4ded-ad99-e43a0197742a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:46.262198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d27aa-d1f9-4ded-ad99-e43a0197742a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:46.266478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d27aa-d1f9-4ded-ad99-e43a0197742a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:46.269393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d27aa-d1f9-4ded-ad99-e43a0197742a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:46.272362 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 133/274] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:46.277942 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d27aa-d1f9-4ded-ad99-e43a0197742a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:46.345973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:46.380303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8d1b8-8623-48b4-9ac0-499280954ba8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:46.419337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d27aa-d1f9-4ded-ad99-e43a0197742a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:46.426215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bee2e-b297-41f2-9a1c-812e62776db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:46.426856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bee2e-b297-41f2-9a1c-812e62776db3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:46.427162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bee2e-b297-41f2-9a1c-812e62776db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:46.427433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bee2e-b297-41f2-9a1c-812e62776db3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:46.440249 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bee2e-b297-41f2-9a1c-812e62776db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-6a7bee2e-b297-41f2-9a1c-812e62776db3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:46.446119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:46.447056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:46.448314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 142/275] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:52:46.454043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bee2e-b297-41f2-9a1c-812e62776db3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:46.465115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:46.471745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82063741-a7c5-4d70-9d30-b996a29222fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:46.472750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063741-a7c5-4d70-9d30-b996a29222fd None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:46.473289 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063741-a7c5-4d70-9d30-b996a29222fd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:46.473794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063741-a7c5-4d70-9d30-b996a29222fd None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:46.474636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82063741-a7c5-4d70-9d30-b996a29222fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:46.476518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82063741-a7c5-4d70-9d30-b996a29222fd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:46.510354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 134/276] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:52:46 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:46.584261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bee2e-b297-41f2-9a1c-812e62776db3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:46.627361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40810ea4-9609-417b-99fe-23136b5ac173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:46.627903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40810ea4-9609-417b-99fe-23136b5ac173 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:46.628183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40810ea4-9609-417b-99fe-23136b5ac173 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:46.628456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40810ea4-9609-417b-99fe-23136b5ac173 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:46.629140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40810ea4-9609-417b-99fe-23136b5ac173 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:46.629445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40810ea4-9609-417b-99fe-23136b5ac173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:46.630849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40810ea4-9609-417b-99fe-23136b5ac173 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:46.641258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 143/277] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:46.646894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82063741-a7c5-4d70-9d30-b996a29222fd None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:46.782297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82063741-a7c5-4d70-9d30-b996a29222fd None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:46.784265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:46.824153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564bfd8-7990-4d82-980e-b47a550caade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:46.825412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564bfd8-7990-4d82-980e-b47a550caade None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:46.825895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564bfd8-7990-4d82-980e-b47a550caade None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:46.826390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564bfd8-7990-4d82-980e-b47a550caade None admin] PATH_INFO: `/v3/users/62e4db66418d4305bb298f4bb94548ba` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:46.834041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dfc82-22c4-420e-b27e-f20e70073a7c None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:46.834534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dfc82-22c4-420e-b27e-f20e70073a7c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:46.834605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dfc82-22c4-420e-b27e-f20e70073a7c None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:46.836229 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 144/278] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:46.842186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564bfd8-7990-4d82-980e-b47a550caade None admin] RBAC: Authorizing `identity:get_user(user_id=62e4db66418d4305bb298f4bb94548ba)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:46.842360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58f5d9-e3cf-4eb3-8170-7a023240da66 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:46.842544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58f5d9-e3cf-4eb3-8170-7a023240da66 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:46.842777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58f5d9-e3cf-4eb3-8170-7a023240da66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:46.843709 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564bfd8-7990-4d82-980e-b47a550caade None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:46.845630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 135/279] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:46 2022] GET /identity/v3/users/62e4db66418d4305bb298f4bb94548ba => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:46.852029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564bfd8-7990-4d82-980e-b47a550caade None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:46.870984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-4a58f5d9-e3cf-4eb3-8170-7a023240da66 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:46.871166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4a58f5d9-e3cf-4eb3-8170-7a023240da66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:46.943531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:46.947484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 145/280] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:46.975431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564bfd8-7990-4d82-980e-b47a550caade None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:47.022471 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36fcd7-d1c8-49f6-a861-7711953dd2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:47.023244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36fcd7-d1c8-49f6-a861-7711953dd2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:47.023441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36fcd7-d1c8-49f6-a861-7711953dd2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:47.023713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36fcd7-d1c8-49f6-a861-7711953dd2b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:47.024506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36fcd7-d1c8-49f6-a861-7711953dd2b5 None admin] RBAC: Adding query filter params (role.id=34ccf81a9f53404e9edefd846827e5b7, user.id=62e4db66418d4305bb298f4bb94548ba, scope.system=all) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:47.024818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36fcd7-d1c8-49f6-a861-7711953dd2b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:47.026316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36fcd7-d1c8-49f6-a861-7711953dd2b5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:47.032563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 136/281] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 22 16:52:46 2022] GET /identity/v3/role_assignments?user.id=62e4db66418d4305bb298f4bb94548ba&scope.system=all&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:47.106130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fadf2-c72b-4145-a38a-9d66e9b0ea03 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:47.106489 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fadf2-c72b-4145-a38a-9d66e9b0ea03 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:47.106709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fadf2-c72b-4145-a38a-9d66e9b0ea03 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:47.108560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 146/282] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:47.114227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de7c1e-0d4f-4d8a-bcc4-bf788f468e62 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:47.114744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de7c1e-0d4f-4d8a-bcc4-bf788f468e62 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:47.114744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de7c1e-0d4f-4d8a-bcc4-bf788f468e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:47.142260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-03de7c1e-0d4f-4d8a-bcc4-bf788f468e62 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:47.143550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-03de7c1e-0d4f-4d8a-bcc4-bf788f468e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:47.173982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:47.233194 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fadf2-c72b-4145-a38a-9d66e9b0ea03 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:47.262221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 137/283] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:47.384621 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fadf2-c72b-4145-a38a-9d66e9b0ea03 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:47.425192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dace19-aa5f-467f-8e3b-28e348583d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:47.425840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dace19-aa5f-467f-8e3b-28e348583d09 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:47.426107 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dace19-aa5f-467f-8e3b-28e348583d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:47.426377 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dace19-aa5f-467f-8e3b-28e348583d09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:47.437937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dace19-aa5f-467f-8e3b-28e348583d09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-f8dace19-aa5f-467f-8e3b-28e348583d09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:47.442967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:47.443867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:47.444653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 147/284] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:52:47.452104 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de7c1e-0d4f-4d8a-bcc4-bf788f468e62 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:47.592226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de7c1e-0d4f-4d8a-bcc4-bf788f468e62 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:47.642366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd150b7a-f4a1-4a2c-8159-1ae2ff634766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:47.643074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd150b7a-f4a1-4a2c-8159-1ae2ff634766 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:47.643357 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd150b7a-f4a1-4a2c-8159-1ae2ff634766 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:47.643630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd150b7a-f4a1-4a2c-8159-1ae2ff634766 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:47.644291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd150b7a-f4a1-4a2c-8159-1ae2ff634766 None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:47.644590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd150b7a-f4a1-4a2c-8159-1ae2ff634766 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:47.645971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd150b7a-f4a1-4a2c-8159-1ae2ff634766 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:47.649860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dace19-aa5f-467f-8e3b-28e348583d09 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:47.659921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 138/285] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:47.664989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd150b7a-f4a1-4a2c-8159-1ae2ff634766 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:47.702065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:47.743394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:47.808152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dace19-aa5f-467f-8e3b-28e348583d09 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:47.867819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd150b7a-f4a1-4a2c-8159-1ae2ff634766 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:47.888124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a084957-f755-4251-93bb-85ea7c8a4f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:47.889487 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a084957-f755-4251-93bb-85ea7c8a4f76 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:47.889956 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a084957-f755-4251-93bb-85ea7c8a4f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:47.890414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a084957-f755-4251-93bb-85ea7c8a4f76 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:47.902524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a084957-f755-4251-93bb-85ea7c8a4f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-5a084957-f755-4251-93bb-85ea7c8a4f76 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:47.908031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:47.908957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:47.909737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:47.917928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 148/286] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:47.920085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a084957-f755-4251-93bb-85ea7c8a4f76 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:47.979818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222c87c-e042-4d85-99de-fd79c5e3d1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:47.980487 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c87c-e042-4d85-99de-fd79c5e3d1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:47.980766 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c87c-e042-4d85-99de-fd79c5e3d1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:47.981034 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c87c-e042-4d85-99de-fd79c5e3d1a5 None admin] PATH_INFO: `/v3/users/fe00075f4d6d453c9a58c46ccc81be6e` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:47.987912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222c87c-e042-4d85-99de-fd79c5e3d1a5 None admin] RBAC: Authorizing `identity:get_user(user_id=fe00075f4d6d453c9a58c46ccc81be6e)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:47.989424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222c87c-e042-4d85-99de-fd79c5e3d1a5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:47.991158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 139/287] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:47 2022] GET /identity/v3/users/fe00075f4d6d453c9a58c46ccc81be6e => generated 297 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:48.011380 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222c87c-e042-4d85-99de-fd79c5e3d1a5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:48.042822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a084957-f755-4251-93bb-85ea7c8a4f76 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:48.063353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:48.084218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574adf2-eebd-4eae-b749-f41e0d9e4bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:48.085278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574adf2-eebd-4eae-b749-f41e0d9e4bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:48.085748 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574adf2-eebd-4eae-b749-f41e0d9e4bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:48.086214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574adf2-eebd-4eae-b749-f41e0d9e4bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:48.087103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574adf2-eebd-4eae-b749-f41e0d9e4bf6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:48.087703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574adf2-eebd-4eae-b749-f41e0d9e4bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:48.089275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574adf2-eebd-4eae-b749-f41e0d9e4bf6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:48.103260 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 149/288] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:48.105108 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574adf2-eebd-4eae-b749-f41e0d9e4bf6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:48.209962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574adf2-eebd-4eae-b749-f41e0d9e4bf6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:48.250995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:48.257592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcbfd5-ba6c-4a4a-9879-d21c637b4943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:48.258539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcbfd5-ba6c-4a4a-9879-d21c637b4943 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:48.259033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcbfd5-ba6c-4a4a-9879-d21c637b4943 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:48.259512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcbfd5-ba6c-4a4a-9879-d21c637b4943 None admin] PATH_INFO: `/v3/users/7d6a80d272564afc95859a3bc9c79c0e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:48.261898 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcbfd5-ba6c-4a4a-9879-d21c637b4943 None admin] RBAC: Authorizing `identity:get_user(user_id=7d6a80d272564afc95859a3bc9c79c0e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:48.263839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcbfd5-ba6c-4a4a-9879-d21c637b4943 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:48.266830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 150/289] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:48 2022] GET /identity/v3/users/7d6a80d272564afc95859a3bc9c79c0e => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:48.274256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcbfd5-ba6c-4a4a-9879-d21c637b4943 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:48.292945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222c87c-e042-4d85-99de-fd79c5e3d1a5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:48.364144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a618a9c-8b9b-496f-b5ba-b56ae9d0ecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:48.365146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a618a9c-8b9b-496f-b5ba-b56ae9d0ecd6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:48.365630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a618a9c-8b9b-496f-b5ba-b56ae9d0ecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:48.366091 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a618a9c-8b9b-496f-b5ba-b56ae9d0ecd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:48.375295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:48.462758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcbfd5-ba6c-4a4a-9879-d21c637b4943 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:48.498390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd50df-7ff9-46ed-884f-088f7e745bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:48.499298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd50df-7ff9-46ed-884f-088f7e745bff None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:48.499753 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd50df-7ff9-46ed-884f-088f7e745bff None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:48.500162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd50df-7ff9-46ed-884f-088f7e745bff None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:48.501494 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd50df-7ff9-46ed-884f-088f7e745bff None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:48.503200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd50df-7ff9-46ed-884f-088f7e745bff None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:48.509848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 151/290] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:48 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:48.511266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd50df-7ff9-46ed-884f-088f7e745bff None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:48.537319 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a618a9c-8b9b-496f-b5ba-b56ae9d0ecd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:48.538839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a618a9c-8b9b-496f-b5ba-b56ae9d0ecd6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:48.621072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd50df-7ff9-46ed-884f-088f7e745bff None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:48.639033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:48.657912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94323606-4c5f-4f5c-a735-4f0336c2530f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:48.658800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323606-4c5f-4f5c-a735-4f0336c2530f None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:48.659264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323606-4c5f-4f5c-a735-4f0336c2530f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:48.659708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323606-4c5f-4f5c-a735-4f0336c2530f None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/users/7d6a80d272564afc95859a3bc9c79c0e/roles/0ab8ef19a25b4e489c9abb312698599c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:48.661804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94323606-4c5f-4f5c-a735-4f0336c2530f None admin] RBAC: Authorizing `identity:create_grant(project_id=c24becb4b27f43ec9eeb7b0ed1066542, user_id=7d6a80d272564afc95859a3bc9c79c0e, role_id=0ab8ef19a25b4e489c9abb312698599c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:48.663457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94323606-4c5f-4f5c-a735-4f0336c2530f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:48.683530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 152/291] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:52:48 2022] PUT /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/users/7d6a80d272564afc95859a3bc9c79c0e/roles/0ab8ef19a25b4e489c9abb312698599c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 16:52:48.742619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:48.771947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 140/292] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 761 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:48.777825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94323606-4c5f-4f5c-a735-4f0336c2530f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:48.908769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94323606-4c5f-4f5c-a735-4f0336c2530f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:48.947502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2d0c2-6814-4c70-a129-a5b4aa7fc9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:48.948374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2d0c2-6814-4c70-a129-a5b4aa7fc9ad None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:48.948812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2d0c2-6814-4c70-a129-a5b4aa7fc9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:48.949217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2d0c2-6814-4c70-a129-a5b4aa7fc9ad None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:48.950578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2d0c2-6814-4c70-a129-a5b4aa7fc9ad None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:48.952203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2d0c2-6814-4c70-a129-a5b4aa7fc9ad None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:48.954055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 153/293] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:48 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:48.960248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a618a9c-8b9b-496f-b5ba-b56ae9d0ecd6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:49.086161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a618a9c-8b9b-496f-b5ba-b56ae9d0ecd6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:49.128737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fe0dd-c18f-42a4-bf65-397f9ddff20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:49.129592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fe0dd-c18f-42a4-bf65-397f9ddff20f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:49.130034 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fe0dd-c18f-42a4-bf65-397f9ddff20f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:49.130444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fe0dd-c18f-42a4-bf65-397f9ddff20f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:49.131334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fe0dd-c18f-42a4-bf65-397f9ddff20f None admin] RBAC: Adding query filter params (role.id=34ccf81a9f53404e9edefd846827e5b7, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542, user.id=fe00075f4d6d453c9a58c46ccc81be6e) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:49.131776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fe0dd-c18f-42a4-bf65-397f9ddff20f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:49.133357 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fe0dd-c18f-42a4-bf65-397f9ddff20f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:49.141934 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 141/294] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:48 2022] GET /identity/v3/role_assignments?user.id=fe00075f4d6d453c9a58c46ccc81be6e&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:49.440100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953304cc-f53b-4d4f-8be8-6b501b7b136f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:49.440687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953304cc-f53b-4d4f-8be8-6b501b7b136f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:49.441122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953304cc-f53b-4d4f-8be8-6b501b7b136f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:49.442864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 154/295] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:49.451471 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595cd42-6ef8-48bd-9846-589f9251fb23 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:49.452489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595cd42-6ef8-48bd-9846-589f9251fb23 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:49.453003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595cd42-6ef8-48bd-9846-589f9251fb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:49.480874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6595cd42-6ef8-48bd-9846-589f9251fb23 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:49.482926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6595cd42-6ef8-48bd-9846-589f9251fb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:49.533470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:49.563061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 142/296] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:49.579034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ab865-01bf-46eb-81ed-18a78707dfa7 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:49.579390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ab865-01bf-46eb-81ed-18a78707dfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:49.579733 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ab865-01bf-46eb-81ed-18a78707dfa7 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:49.581178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 155/297] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:49.591016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccc2de-c97b-4c18-a52b-8bbc23c90c02 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:49.591016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccc2de-c97b-4c18-a52b-8bbc23c90c02 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:49.591016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccc2de-c97b-4c18-a52b-8bbc23c90c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:49.614352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0eccc2de-c97b-4c18-a52b-8bbc23c90c02 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:49.615844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0eccc2de-c97b-4c18-a52b-8bbc23c90c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:49.693093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 143/298] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:49.713746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb45bd5-8461-4213-83a9-37c1a5bac45b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:49.714125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb45bd5-8461-4213-83a9-37c1a5bac45b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:49.714405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb45bd5-8461-4213-83a9-37c1a5bac45b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:49.717408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 156/299] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:49.722296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072c45a-d411-4b50-b234-e8a85c644893 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:49.722793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072c45a-d411-4b50-b234-e8a85c644893 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:49.723223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072c45a-d411-4b50-b234-e8a85c644893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:49.744217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f072c45a-d411-4b50-b234-e8a85c644893 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:49.745472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f072c45a-d411-4b50-b234-e8a85c644893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:49.831435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 144/300] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:49.881165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb45bd5-8461-4213-83a9-37c1a5bac45b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:49.932485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:49.991886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072c45a-d411-4b50-b234-e8a85c644893 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:49.994080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb45bd5-8461-4213-83a9-37c1a5bac45b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:50.032420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4edab-2c10-4d81-9889-639ce9cb8135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:50.033163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4edab-2c10-4d81-9889-639ce9cb8135 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:50.033443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4edab-2c10-4d81-9889-639ce9cb8135 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:50.033709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4edab-2c10-4d81-9889-639ce9cb8135 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:50.045282 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:50.047657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4edab-2c10-4d81-9889-639ce9cb8135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e9d4edab-2c10-4d81-9889-639ce9cb8135 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:50.055067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:50.056127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:50.057051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:50.059854 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 157/301] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:50.066835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4edab-2c10-4d81-9889-639ce9cb8135 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:50.117365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072c45a-d411-4b50-b234-e8a85c644893 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:50.154483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8c696-f7a6-4bbf-9553-5638b02f4bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:50.155150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8c696-f7a6-4bbf-9553-5638b02f4bdf None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:50.155434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8c696-f7a6-4bbf-9553-5638b02f4bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:50.155707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8c696-f7a6-4bbf-9553-5638b02f4bdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:50.156821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8c696-f7a6-4bbf-9553-5638b02f4bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:50.158801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8c696-f7a6-4bbf-9553-5638b02f4bdf None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:50.161370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 145/302] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:50.162180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8c696-f7a6-4bbf-9553-5638b02f4bdf None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:50.185441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4edab-2c10-4d81-9889-639ce9cb8135 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:50.188959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:50.223341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5903d4-918a-4c97-b9e4-3decb341e959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:50.223945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5903d4-918a-4c97-b9e4-3decb341e959 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:50.224236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5903d4-918a-4c97-b9e4-3decb341e959 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:50.224490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5903d4-918a-4c97-b9e4-3decb341e959 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:50.225146 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5903d4-918a-4c97-b9e4-3decb341e959 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:50.225444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5903d4-918a-4c97-b9e4-3decb341e959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:50.226823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5903d4-918a-4c97-b9e4-3decb341e959 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:50.235867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 158/303] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:50.236593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5903d4-918a-4c97-b9e4-3decb341e959 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:50.273158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8c696-f7a6-4bbf-9553-5638b02f4bdf None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:50.281251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:50.314400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0262cf-842e-48ae-ab35-0cbbf5cab78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:50.315059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0262cf-842e-48ae-ab35-0cbbf5cab78d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:50.315340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0262cf-842e-48ae-ab35-0cbbf5cab78d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:50.315611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0262cf-842e-48ae-ab35-0cbbf5cab78d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:50.326629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0262cf-842e-48ae-ab35-0cbbf5cab78d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-6b0262cf-842e-48ae-ab35-0cbbf5cab78d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:50.331448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:50.332297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 146/304] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:50.335846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0262cf-842e-48ae-ab35-0cbbf5cab78d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:50.360227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5903d4-918a-4c97-b9e4-3decb341e959 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:50.395288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df68f8-dcba-4bb0-a66a-daa3a4a8265f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:50.395903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df68f8-dcba-4bb0-a66a-daa3a4a8265f None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:50.396186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df68f8-dcba-4bb0-a66a-daa3a4a8265f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:50.396459 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df68f8-dcba-4bb0-a66a-daa3a4a8265f None admin] PATH_INFO: `/v3/groups` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:50.397094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df68f8-dcba-4bb0-a66a-daa3a4a8265f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:50.398519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df68f8-dcba-4bb0-a66a-daa3a4a8265f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:50.415499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 159/305] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Sep 22 16:52:50 2022] POST /identity/v3/groups => generated 229 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:52:50.416325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df68f8-dcba-4bb0-a66a-daa3a4a8265f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:50.456488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0262cf-842e-48ae-ab35-0cbbf5cab78d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:50.458038 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:50.513296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737cdae-1ab9-4da4-8420-f8a0c16ef7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:50.514980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737cdae-1ab9-4da4-8420-f8a0c16ef7cd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:50.515743 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737cdae-1ab9-4da4-8420-f8a0c16ef7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:50.516207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737cdae-1ab9-4da4-8420-f8a0c16ef7cd None admin] PATH_INFO: `/v3/users/b963460aced047ed90ee5ccfbe7bff9c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:50.537935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737cdae-1ab9-4da4-8420-f8a0c16ef7cd None admin] RBAC: Authorizing `identity:get_user(user_id=b963460aced047ed90ee5ccfbe7bff9c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:50.541176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737cdae-1ab9-4da4-8420-f8a0c16ef7cd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:50.544651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 147/306] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:50 2022] GET /identity/v3/users/b963460aced047ed90ee5ccfbe7bff9c => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:50.551100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737cdae-1ab9-4da4-8420-f8a0c16ef7cd None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:50.554772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df68f8-dcba-4bb0-a66a-daa3a4a8265f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:50.580134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:50.610702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6cee9-b444-4c5a-b831-ba41d4d69a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:50.611754 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6cee9-b444-4c5a-b831-ba41d4d69a79 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:50.612170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6cee9-b444-4c5a-b831-ba41d4d69a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:50.612572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6cee9-b444-4c5a-b831-ba41d4d69a79 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:50.614861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6cee9-b444-4c5a-b831-ba41d4d69a79 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:50.615635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6cee9-b444-4c5a-b831-ba41d4d69a79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:50.618583 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6cee9-b444-4c5a-b831-ba41d4d69a79 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:50.631209 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 160/307] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:50.638558 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6cee9-b444-4c5a-b831-ba41d4d69a79 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:50.678032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737cdae-1ab9-4da4-8420-f8a0c16ef7cd None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:50.714545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dee9b-c7f9-4a6e-9078-32ce52472d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:50.715284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dee9b-c7f9-4a6e-9078-32ce52472d57 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:50.715730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dee9b-c7f9-4a6e-9078-32ce52472d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:50.716096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dee9b-c7f9-4a6e-9078-32ce52472d57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:50.716893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dee9b-c7f9-4a6e-9078-32ce52472d57 None admin] RBAC: Adding query filter params (role.id=0ab8ef19a25b4e489c9abb312698599c, user.id=b963460aced047ed90ee5ccfbe7bff9c, scope.system=all) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:50.717266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dee9b-c7f9-4a6e-9078-32ce52472d57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:50.718889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dee9b-c7f9-4a6e-9078-32ce52472d57 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:50.724735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 148/308] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 22 16:52:50 2022] GET /identity/v3/role_assignments?user.id=b963460aced047ed90ee5ccfbe7bff9c&scope.system=all&role.id=0ab8ef19a25b4e489c9abb312698599c => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:50.764537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:50.783543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6cee9-b444-4c5a-b831-ba41d4d69a79 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:50.824594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6284e-b7ce-4a86-a5fb-0537963f64d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:50.825767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6284e-b7ce-4a86-a5fb-0537963f64d1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:50.826056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6284e-b7ce-4a86-a5fb-0537963f64d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:50.826330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6284e-b7ce-4a86-a5fb-0537963f64d1 None admin] PATH_INFO: `/v3/users/62e4db66418d4305bb298f4bb94548ba` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:50.827640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6284e-b7ce-4a86-a5fb-0537963f64d1 None admin] RBAC: Authorizing `identity:get_user(user_id=62e4db66418d4305bb298f4bb94548ba)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:50.829138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6284e-b7ce-4a86-a5fb-0537963f64d1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:50.831274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 161/309] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:50 2022] GET /identity/v3/users/62e4db66418d4305bb298f4bb94548ba => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:50.837129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dee9b-c7f9-4a6e-9078-32ce52472d57 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:50.938413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:50.957431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dee9b-c7f9-4a6e-9078-32ce52472d57 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:50.993976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2498d31-38a4-4065-9a6d-a37b554b3288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:50.995033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498d31-38a4-4065-9a6d-a37b554b3288 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:50.995521 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498d31-38a4-4065-9a6d-a37b554b3288 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:50.995994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498d31-38a4-4065-9a6d-a37b554b3288 None admin] PATH_INFO: `/v3/system/users/62e4db66418d4305bb298f4bb94548ba/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:50.997760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2498d31-38a4-4065-9a6d-a37b554b3288 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=62e4db66418d4305bb298f4bb94548ba, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:50.999436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2498d31-38a4-4065-9a6d-a37b554b3288 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:51.010422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 149/310] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Sep 22 16:52:50 2022] PUT /identity/v3/system/users/62e4db66418d4305bb298f4bb94548ba/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:52:51.417285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a97a6a-ce26-4fc6-99b9-3f715762af0e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:51.418318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a97a6a-ce26-4fc6-99b9-3f715762af0e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:51.419398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a97a6a-ce26-4fc6-99b9-3f715762af0e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:51.421264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 162/311] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:51.427324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f37bc-8aa8-46d5-942d-d699e73d2b12 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:51.427903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f37bc-8aa8-46d5-942d-d699e73d2b12 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:51.428376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f37bc-8aa8-46d5-942d-d699e73d2b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:51.448667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9b1f37bc-8aa8-46d5-942d-d699e73d2b12 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:51.450879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9b1f37bc-8aa8-46d5-942d-d699e73d2b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:51.524579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 150/312] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:51.808489 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a97a6a-ce26-4fc6-99b9-3f715762af0e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:51.917269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a97a6a-ce26-4fc6-99b9-3f715762af0e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:51.954848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab8a04-335e-4998-82af-dcdba5fbcd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:51.955518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab8a04-335e-4998-82af-dcdba5fbcd16 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:51.955800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab8a04-335e-4998-82af-dcdba5fbcd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:51.956077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab8a04-335e-4998-82af-dcdba5fbcd16 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:51.967389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab8a04-335e-4998-82af-dcdba5fbcd16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-99ab8a04-335e-4998-82af-dcdba5fbcd16 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:51.972692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:51.973546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:51.974327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 163/313] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:51.981774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f37bc-8aa8-46d5-942d-d699e73d2b12 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:52.066979 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:52.087370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f37bc-8aa8-46d5-942d-d699e73d2b12 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:52.130996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f6645-31c3-4053-a8ac-bd8ecf8b0bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:52.131613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f6645-31c3-4053-a8ac-bd8ecf8b0bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:52.131896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f6645-31c3-4053-a8ac-bd8ecf8b0bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:52.132167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f6645-31c3-4053-a8ac-bd8ecf8b0bd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:52.132828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f6645-31c3-4053-a8ac-bd8ecf8b0bd2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:52.133129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f6645-31c3-4053-a8ac-bd8ecf8b0bd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:52.134568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f6645-31c3-4053-a8ac-bd8ecf8b0bd2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:52.143852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 151/314] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:52.149443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab8a04-335e-4998-82af-dcdba5fbcd16 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:52.169008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:52.262034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab8a04-335e-4998-82af-dcdba5fbcd16 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:52.298763 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abea5e9-994f-460f-8d11-44d751a70712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:52.299836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abea5e9-994f-460f-8d11-44d751a70712 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:52.300336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abea5e9-994f-460f-8d11-44d751a70712 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:52.300839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abea5e9-994f-460f-8d11-44d751a70712 None admin] PATH_INFO: `/v3/users/7d6a80d272564afc95859a3bc9c79c0e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:52.302300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abea5e9-994f-460f-8d11-44d751a70712 None admin] RBAC: Authorizing `identity:get_user(user_id=7d6a80d272564afc95859a3bc9c79c0e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:52.303997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abea5e9-994f-460f-8d11-44d751a70712 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:52.306362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 164/315] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:52 2022] GET /identity/v3/users/7d6a80d272564afc95859a3bc9c79c0e => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:52.312435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f6645-31c3-4053-a8ac-bd8ecf8b0bd2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:52.431275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f6645-31c3-4053-a8ac-bd8ecf8b0bd2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:52.467422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5eabe-e26c-4a38-8a3e-cd8653d4d432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:52.468384 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5eabe-e26c-4a38-8a3e-cd8653d4d432 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:52.468863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5eabe-e26c-4a38-8a3e-cd8653d4d432 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:52.469327 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5eabe-e26c-4a38-8a3e-cd8653d4d432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:52.486269 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:52.551175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5eabe-e26c-4a38-8a3e-cd8653d4d432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:52.552757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5eabe-e26c-4a38-8a3e-cd8653d4d432 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:52.663233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 152/316] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:52.672621 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abea5e9-994f-460f-8d11-44d751a70712 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:52.787505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:52.800729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abea5e9-994f-460f-8d11-44d751a70712 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:52.841536 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bc797-e47d-4781-8063-f25f0fc4b0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:52.842531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bc797-e47d-4781-8063-f25f0fc4b0cf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:52.843035 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bc797-e47d-4781-8063-f25f0fc4b0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:52.843521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bc797-e47d-4781-8063-f25f0fc4b0cf None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:52.844948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bc797-e47d-4781-8063-f25f0fc4b0cf None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:52.847736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bc797-e47d-4781-8063-f25f0fc4b0cf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:52.849961 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 165/317] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:52 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:52.859852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5eabe-e26c-4a38-8a3e-cd8653d4d432 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:52.878917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea8f86-8ea8-46bb-a38b-3636b9fae89a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:52.879678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea8f86-8ea8-46bb-a38b-3636b9fae89a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:52.880167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea8f86-8ea8-46bb-a38b-3636b9fae89a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:52.881995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 166/318] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:52.889648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce4f3b-d555-49c7-a574-a3127612b56e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:52.890298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce4f3b-d555-49c7-a574-a3127612b56e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:52.890871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce4f3b-d555-49c7-a574-a3127612b56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:52.912968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1fce4f3b-d555-49c7-a574-a3127612b56e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:52.914385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1fce4f3b-d555-49c7-a574-a3127612b56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:52.925167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:52.992677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5eabe-e26c-4a38-8a3e-cd8653d4d432 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:52.996789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 167/319] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:52.997875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378d712-6e63-4092-919c-0eb2b9b8f3d7 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:52.998159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378d712-6e63-4092-919c-0eb2b9b8f3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:52.998431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378d712-6e63-4092-919c-0eb2b9b8f3d7 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:53.006993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 168/320] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:53.006993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67324deb-1ea9-4754-8557-01cdf76b6443 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:53.006993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67324deb-1ea9-4754-8557-01cdf76b6443 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:53.006993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67324deb-1ea9-4754-8557-01cdf76b6443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:53.026363 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-67324deb-1ea9-4754-8557-01cdf76b6443 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:53.027691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-67324deb-1ea9-4754-8557-01cdf76b6443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:53.033020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bdb4a3-df74-45e6-918f-ba46e958821b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:53.033645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdb4a3-df74-45e6-918f-ba46e958821b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:53.033967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdb4a3-df74-45e6-918f-ba46e958821b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:53.034243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdb4a3-df74-45e6-918f-ba46e958821b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:53.034987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bdb4a3-df74-45e6-918f-ba46e958821b None admin] RBAC: Adding query filter params (role.id=0ab8ef19a25b4e489c9abb312698599c, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542, user.id=7d6a80d272564afc95859a3bc9c79c0e) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:53.035298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bdb4a3-df74-45e6-918f-ba46e958821b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:53.036727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bdb4a3-df74-45e6-918f-ba46e958821b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:53.043243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 153/321] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:52:52 2022] GET /identity/v3/role_assignments?user.id=7d6a80d272564afc95859a3bc9c79c0e&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=0ab8ef19a25b4e489c9abb312698599c => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:53.103301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 169/322] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:53.284958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bdb4a3-df74-45e6-918f-ba46e958821b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:53.384698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67324deb-1ea9-4754-8557-01cdf76b6443 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:53.399244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:53.401827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bdb4a3-df74-45e6-918f-ba46e958821b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:53.411585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:53.436712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6ca92-dbe6-4e11-b920-ab1e121b78aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:53.437333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6ca92-dbe6-4e11-b920-ab1e121b78aa None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:53.437608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6ca92-dbe6-4e11-b920-ab1e121b78aa None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:53.437877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6ca92-dbe6-4e11-b920-ab1e121b78aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:53.438974 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6ca92-dbe6-4e11-b920-ab1e121b78aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:53.440427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6ca92-dbe6-4e11-b920-ab1e121b78aa None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:53.441969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 154/323] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:52:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:53.443162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2992881-dee9-41ac-9b94-a1429fe80b6e None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:53.443380 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2992881-dee9-41ac-9b94-a1429fe80b6e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:53.443633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2992881-dee9-41ac-9b94-a1429fe80b6e None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:53.444934 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 155/324] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:53.447368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2992881-dee9-41ac-9b94-a1429fe80b6e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:53.490112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67324deb-1ea9-4754-8557-01cdf76b6443 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:53.524297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450e137-a257-4e6f-8e77-9f507ff96c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:53.524930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450e137-a257-4e6f-8e77-9f507ff96c04 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:53.525209 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450e137-a257-4e6f-8e77-9f507ff96c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:53.525482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450e137-a257-4e6f-8e77-9f507ff96c04 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:53.536568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450e137-a257-4e6f-8e77-9f507ff96c04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a450e137-a257-4e6f-8e77-9f507ff96c04 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:53.541381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:53.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:53.543405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 170/325] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:52:53.544143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda42607-9e9a-44bb-a1cc-004f1b2cf012 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:53.544143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda42607-9e9a-44bb-a1cc-004f1b2cf012 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:53.544370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda42607-9e9a-44bb-a1cc-004f1b2cf012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:53.559933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2992881-dee9-41ac-9b94-a1429fe80b6e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:53.566240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-fda42607-9e9a-44bb-a1cc-004f1b2cf012 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:53.567478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fda42607-9e9a-44bb-a1cc-004f1b2cf012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:53.593130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7413e9b-5acb-4fce-ad2b-00915bb8f58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:53.593740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413e9b-5acb-4fce-ad2b-00915bb8f58e None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:53.594018 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413e9b-5acb-4fce-ad2b-00915bb8f58e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:53.594289 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413e9b-5acb-4fce-ad2b-00915bb8f58e None admin] PATH_INFO: `/v3/groups` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:53.594918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7413e9b-5acb-4fce-ad2b-00915bb8f58e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:53.596376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7413e9b-5acb-4fce-ad2b-00915bb8f58e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:53.614007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 156/326] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Sep 22 16:52:53 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 16:52:53.617196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7413e9b-5acb-4fce-ad2b-00915bb8f58e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:53.648892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 171/327] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:53.678545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:53.759969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7413e9b-5acb-4fce-ad2b-00915bb8f58e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:53.802100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e774d-bb4a-4bd1-9782-aef77790ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:53.803008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e774d-bb4a-4bd1-9782-aef77790ecff None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:53.803008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e774d-bb4a-4bd1-9782-aef77790ecff None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:53.803192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e774d-bb4a-4bd1-9782-aef77790ecff None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:53.803857 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e774d-bb4a-4bd1-9782-aef77790ecff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:53.804164 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e774d-bb4a-4bd1-9782-aef77790ecff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:53.805535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e774d-bb4a-4bd1-9782-aef77790ecff None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:53.816365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 157/328] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:53.822077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda42607-9e9a-44bb-a1cc-004f1b2cf012 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:53.825328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:53.918401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e774d-bb4a-4bd1-9782-aef77790ecff None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:53.943779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda42607-9e9a-44bb-a1cc-004f1b2cf012 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:53.992726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce8b05-9281-4b49-89f8-ceedca4431c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:53.993858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8b05-9281-4b49-89f8-ceedca4431c7 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:53.994162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8b05-9281-4b49-89f8-ceedca4431c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:53.994433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8b05-9281-4b49-89f8-ceedca4431c7 None admin] PATH_INFO: `/v3/users/b963460aced047ed90ee5ccfbe7bff9c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:53.995943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce8b05-9281-4b49-89f8-ceedca4431c7 None admin] RBAC: Authorizing `identity:get_user(user_id=b963460aced047ed90ee5ccfbe7bff9c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:53.998267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce8b05-9281-4b49-89f8-ceedca4431c7 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:54.000740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 172/329] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:53 2022] GET /identity/v3/users/b963460aced047ed90ee5ccfbe7bff9c => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:54.006389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce8b05-9281-4b49-89f8-ceedca4431c7 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:54.065210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e774d-bb4a-4bd1-9782-aef77790ecff None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:54.067709 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:54.108656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:54.119071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979ad04-e847-496c-8537-7f38dfbe2f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:54.119724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979ad04-e847-496c-8537-7f38dfbe2f2d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:54.120001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979ad04-e847-496c-8537-7f38dfbe2f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:54.120270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979ad04-e847-496c-8537-7f38dfbe2f2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:54.134826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979ad04-e847-496c-8537-7f38dfbe2f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3979ad04-e847-496c-8537-7f38dfbe2f2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:54.140409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:54.141426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:54.142262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 158/330] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:52:54.149800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979ad04-e847-496c-8537-7f38dfbe2f2d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:54.154658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce8b05-9281-4b49-89f8-ceedca4431c7 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:54.194636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d44e6-7226-4475-b991-cbb4e146e6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:54.195404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d44e6-7226-4475-b991-cbb4e146e6ef None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:54.195872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d44e6-7226-4475-b991-cbb4e146e6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:54.196156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d44e6-7226-4475-b991-cbb4e146e6ef None admin] PATH_INFO: `/v3/system/users/b963460aced047ed90ee5ccfbe7bff9c/roles/0ab8ef19a25b4e489c9abb312698599c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:54.197807 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d44e6-7226-4475-b991-cbb4e146e6ef None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b963460aced047ed90ee5ccfbe7bff9c, role_id=0ab8ef19a25b4e489c9abb312698599c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:54.199432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d44e6-7226-4475-b991-cbb4e146e6ef None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:54.216926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 173/331] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Sep 22 16:52:54 2022] PUT /identity/v3/system/users/b963460aced047ed90ee5ccfbe7bff9c/roles/0ab8ef19a25b4e489c9abb312698599c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 16:52:54.287654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979ad04-e847-496c-8537-7f38dfbe2f2d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:54.334257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7d796-c8f3-49e2-8212-983079824fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:54.334890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7d796-c8f3-49e2-8212-983079824fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:54.335363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7d796-c8f3-49e2-8212-983079824fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:54.335653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7d796-c8f3-49e2-8212-983079824fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:54.336495 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7d796-c8f3-49e2-8212-983079824fc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:54.336844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7d796-c8f3-49e2-8212-983079824fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:54.338798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7d796-c8f3-49e2-8212-983079824fc4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:54.350037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 159/332] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:54.356762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d44e6-7226-4475-b991-cbb4e146e6ef None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:54.505813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d44e6-7226-4475-b991-cbb4e146e6ef None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:54.545606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8c876-6d29-4c83-bffc-708dc48687ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:54.546288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c876-6d29-4c83-bffc-708dc48687ae None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:54.546596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c876-6d29-4c83-bffc-708dc48687ae None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:54.546878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c876-6d29-4c83-bffc-708dc48687ae None admin] PATH_INFO: `/v3/users/62e4db66418d4305bb298f4bb94548ba` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:54.548132 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8c876-6d29-4c83-bffc-708dc48687ae None admin] RBAC: Authorizing `identity:get_user(user_id=62e4db66418d4305bb298f4bb94548ba)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:54.549864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8c876-6d29-4c83-bffc-708dc48687ae None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:54.551802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 174/333] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:54 2022] GET /identity/v3/users/62e4db66418d4305bb298f4bb94548ba => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:52:54.557393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7d796-c8f3-49e2-8212-983079824fc4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:54.610188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:54.720420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7d796-c8f3-49e2-8212-983079824fc4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:54.771762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bde58-12d0-43fd-9240-7b35cb5f61bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:54.772393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bde58-12d0-43fd-9240-7b35cb5f61bd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:54.772677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bde58-12d0-43fd-9240-7b35cb5f61bd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:54.772950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bde58-12d0-43fd-9240-7b35cb5f61bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:54.773682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bde58-12d0-43fd-9240-7b35cb5f61bd None admin] RBAC: Adding query filter params (role.id=34ccf81a9f53404e9edefd846827e5b7, user.id=62e4db66418d4305bb298f4bb94548ba, scope.system=all) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:54.773988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bde58-12d0-43fd-9240-7b35cb5f61bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:54.775476 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bde58-12d0-43fd-9240-7b35cb5f61bd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:54.789282 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 160/334] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 22 16:52:54 2022] GET /identity/v3/role_assignments?user.id=62e4db66418d4305bb298f4bb94548ba&scope.system=all&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 510 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:56.623931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b2cb0-fcb3-4f68-8c53-c21013411562 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:56.624457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b2cb0-fcb3-4f68-8c53-c21013411562 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:56.624457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b2cb0-fcb3-4f68-8c53-c21013411562 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:56.626080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 175/335] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:56.631905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88ad5b-b6cc-4e10-8356-ed1caceea0e2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:56.632893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88ad5b-b6cc-4e10-8356-ed1caceea0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:56.633359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88ad5b-b6cc-4e10-8356-ed1caceea0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:56.653609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ea88ad5b-b6cc-4e10-8356-ed1caceea0e2 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:56.655141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ea88ad5b-b6cc-4e10-8356-ed1caceea0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:56.739615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 161/336] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:56.748839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091388b-0e33-4a5c-b0a5-c2faaeb0706b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:56.751505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091388b-0e33-4a5c-b0a5-c2faaeb0706b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:56.751505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091388b-0e33-4a5c-b0a5-c2faaeb0706b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:56.751505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 176/337] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:56.757088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90ffe-2f94-4836-8b1d-a88f23ea9eea None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:56.757651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90ffe-2f94-4836-8b1d-a88f23ea9eea None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:56.758135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90ffe-2f94-4836-8b1d-a88f23ea9eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:56.791980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-00e90ffe-2f94-4836-8b1d-a88f23ea9eea None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:56.794036 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-00e90ffe-2f94-4836-8b1d-a88f23ea9eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:56.796456 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:56.868586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 162/338] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:57.116014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091388b-0e33-4a5c-b0a5-c2faaeb0706b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:57.119261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:57.255573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091388b-0e33-4a5c-b0a5-c2faaeb0706b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:57.299027 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524f327-8cb1-4bb0-93fd-b3da5fda0d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:57.299659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524f327-8cb1-4bb0-93fd-b3da5fda0d3c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:57.299934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524f327-8cb1-4bb0-93fd-b3da5fda0d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:57.300202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524f327-8cb1-4bb0-93fd-b3da5fda0d3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:57.311904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e524f327-8cb1-4bb0-93fd-b3da5fda0d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e524f327-8cb1-4bb0-93fd-b3da5fda0d3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:57.318659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:57.319576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:57.320580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 177/339] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:57.327452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e90ffe-2f94-4836-8b1d-a88f23ea9eea None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:57.407816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea1d81-9d1a-4117-9f60-a1ffe5ba9e84 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:57.408093 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea1d81-9d1a-4117-9f60-a1ffe5ba9e84 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:57.408282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea1d81-9d1a-4117-9f60-a1ffe5ba9e84 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:57.409608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 178/340] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:57.417150 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847f7e-31c5-40cd-9eb9-7d7131b0ca6f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:57.417655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847f7e-31c5-40cd-9eb9-7d7131b0ca6f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:57.418902 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847f7e-31c5-40cd-9eb9-7d7131b0ca6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:57.451962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-9c847f7e-31c5-40cd-9eb9-7d7131b0ca6f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:57.453142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9c847f7e-31c5-40cd-9eb9-7d7131b0ca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:57.483880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e90ffe-2f94-4836-8b1d-a88f23ea9eea None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:57.522645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b78d68-7634-431f-b6b2-f1a6d086b412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:57.523549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b78d68-7634-431f-b6b2-f1a6d086b412 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:57.524035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b78d68-7634-431f-b6b2-f1a6d086b412 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:57.524482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b78d68-7634-431f-b6b2-f1a6d086b412 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:57.525338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b78d68-7634-431f-b6b2-f1a6d086b412 None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:57.525781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b78d68-7634-431f-b6b2-f1a6d086b412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:57.527417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b78d68-7634-431f-b6b2-f1a6d086b412 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:57.533216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:57.538449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 163/341] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:57.540955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b78d68-7634-431f-b6b2-f1a6d086b412 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:57.567216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:57.572693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 179/342] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:52:57.573568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c847f7e-31c5-40cd-9eb9-7d7131b0ca6f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:57.659959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b78d68-7634-431f-b6b2-f1a6d086b412 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:57.706286 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c2633-2a10-4e95-8774-33e2e8125a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:57.707210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c2633-2a10-4e95-8774-33e2e8125a5e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:57.707640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c2633-2a10-4e95-8774-33e2e8125a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:57.708098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c2633-2a10-4e95-8774-33e2e8125a5e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:57.709771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c847f7e-31c5-40cd-9eb9-7d7131b0ca6f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:57.726964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c2633-2a10-4e95-8774-33e2e8125a5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-452c2633-2a10-4e95-8774-33e2e8125a5e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:57.738402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:57.739544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:57.740369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:57.742533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 164/343] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:52:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:57.752184 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c2633-2a10-4e95-8774-33e2e8125a5e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:57.796312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903cb56-f6ac-47bb-b172-ea155ad9d6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:57.796951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903cb56-f6ac-47bb-b172-ea155ad9d6ce None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:57.797235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903cb56-f6ac-47bb-b172-ea155ad9d6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:57.797509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903cb56-f6ac-47bb-b172-ea155ad9d6ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:57.847748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:57.889472 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:57.909456 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c2633-2a10-4e95-8774-33e2e8125a5e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:57.917932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903cb56-f6ac-47bb-b172-ea155ad9d6ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:57.920068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903cb56-f6ac-47bb-b172-ea155ad9d6ce None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:57.959916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc97eba-ecbe-4b10-b40e-b76f6c2569f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:57.960546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc97eba-ecbe-4b10-b40e-b76f6c2569f7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:57.960827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc97eba-ecbe-4b10-b40e-b76f6c2569f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:57.961097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc97eba-ecbe-4b10-b40e-b76f6c2569f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:57.961846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc97eba-ecbe-4b10-b40e-b76f6c2569f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:57.962155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc97eba-ecbe-4b10-b40e-b76f6c2569f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:57.963607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc97eba-ecbe-4b10-b40e-b76f6c2569f7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:57.973589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 165/344] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:52:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:57.975069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b911bbd-7ae7-4c0c-b090-ea20d453a119 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:57.975281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b911bbd-7ae7-4c0c-b090-ea20d453a119 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:57.975549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b911bbd-7ae7-4c0c-b090-ea20d453a119 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:57.977116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 166/345] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:57.977843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b911bbd-7ae7-4c0c-b090-ea20d453a119 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:58.093768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 180/346] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:58.095148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3cdf7-6cb9-45b0-9316-4fae46294746 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:58.095433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3cdf7-6cb9-45b0-9316-4fae46294746 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:58.095706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3cdf7-6cb9-45b0-9316-4fae46294746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:58.102669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b911bbd-7ae7-4c0c-b090-ea20d453a119 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:58.124531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b7d3cdf7-6cb9-45b0-9316-4fae46294746 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:58.126376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b7d3cdf7-6cb9-45b0-9316-4fae46294746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:58.142812 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ff984-22c2-4f6a-bbae-b18e425aa260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:58.143498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ff984-22c2-4f6a-bbae-b18e425aa260 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:58.143778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ff984-22c2-4f6a-bbae-b18e425aa260 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:58.144048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ff984-22c2-4f6a-bbae-b18e425aa260 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:58.144886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:58.161435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ff984-22c2-4f6a-bbae-b18e425aa260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-122ff984-22c2-4f6a-bbae-b18e425aa260 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:58.166496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:58.167571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:52:58.168517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 167/347] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:58.169226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ff984-22c2-4f6a-bbae-b18e425aa260 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:58.222439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 181/348] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:58.223595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3cdf7-6cb9-45b0-9316-4fae46294746 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:58.255748 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:58.336879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ff984-22c2-4f6a-bbae-b18e425aa260 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:58.372977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3cdf7-6cb9-45b0-9316-4fae46294746 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:58.387590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc467e-7566-4bba-8a23-7f31a427bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:58.388915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc467e-7566-4bba-8a23-7f31a427bafe None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:58.389214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc467e-7566-4bba-8a23-7f31a427bafe None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:58.389490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc467e-7566-4bba-8a23-7f31a427bafe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:58.437796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba3507-6511-423f-a9b8-996dfc55e5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:58.439649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba3507-6511-423f-a9b8-996dfc55e5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:58.440633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba3507-6511-423f-a9b8-996dfc55e5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:58.442025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba3507-6511-423f-a9b8-996dfc55e5d6 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:58.444241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba3507-6511-423f-a9b8-996dfc55e5d6 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:58.445926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba3507-6511-423f-a9b8-996dfc55e5d6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:58.448794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 182/349] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:58 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:58.453237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba3507-6511-423f-a9b8-996dfc55e5d6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:58.486223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:58.527287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc467e-7566-4bba-8a23-7f31a427bafe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:58.529803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc467e-7566-4bba-8a23-7f31a427bafe None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:58.612830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:58.630835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba3507-6511-423f-a9b8-996dfc55e5d6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:58.676186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad358d-13e9-4852-bfb5-7af619f81fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:58.676811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad358d-13e9-4852-bfb5-7af619f81fde None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:58.677140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad358d-13e9-4852-bfb5-7af619f81fde None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:58.677457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad358d-13e9-4852-bfb5-7af619f81fde None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:58.678739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad358d-13e9-4852-bfb5-7af619f81fde None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:58.679071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad358d-13e9-4852-bfb5-7af619f81fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:58.680460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad358d-13e9-4852-bfb5-7af619f81fde None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:58.681169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 168/350] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:52:58.684146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc467e-7566-4bba-8a23-7f31a427bafe None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:58.690883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 183/351] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:58.692101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d1c70-4d8a-4579-aa44-6e05c238bb98 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:58.692386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d1c70-4d8a-4579-aa44-6e05c238bb98 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:58.692662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d1c70-4d8a-4579-aa44-6e05c238bb98 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:58.694458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 184/352] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:58.694825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d1c70-4d8a-4579-aa44-6e05c238bb98 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:58.799484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:58.823322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc467e-7566-4bba-8a23-7f31a427bafe None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:58.830702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d1c70-4d8a-4579-aa44-6e05c238bb98 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:58.877999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558185c-93ea-4b69-bdbd-cdcedf6495f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:58.878618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558185c-93ea-4b69-bdbd-cdcedf6495f7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:58.878900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558185c-93ea-4b69-bdbd-cdcedf6495f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:58.879224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558185c-93ea-4b69-bdbd-cdcedf6495f7 None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:58.881201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080858e5-1138-4aa0-ac69-fc578c5b1e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:58.882060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080858e5-1138-4aa0-ac69-fc578c5b1e2c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:58.882339 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080858e5-1138-4aa0-ac69-fc578c5b1e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:58.882607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080858e5-1138-4aa0-ac69-fc578c5b1e2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:58.886333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558185c-93ea-4b69-bdbd-cdcedf6495f7 None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:58.888545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558185c-93ea-4b69-bdbd-cdcedf6495f7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:58.890479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 169/353] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:52:58 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:58.891112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558185c-93ea-4b69-bdbd-cdcedf6495f7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:58.897606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080858e5-1138-4aa0-ac69-fc578c5b1e2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-080858e5-1138-4aa0-ac69-fc578c5b1e2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:58.903323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:58.904284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:52:58.905083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:58.905913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 185/354] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:52:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:58.910245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d9e0b-b44b-47fc-a430-ff21300ee23e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:58.910245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d9e0b-b44b-47fc-a430-ff21300ee23e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:58.910245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d9e0b-b44b-47fc-a430-ff21300ee23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:58.929254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-324d9e0b-b44b-47fc-a430-ff21300ee23e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:58.930499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-324d9e0b-b44b-47fc-a430-ff21300ee23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:58.940460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:59.012753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558185c-93ea-4b69-bdbd-cdcedf6495f7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:59.024514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 186/355] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:59.026755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d9e0b-b44b-47fc-a430-ff21300ee23e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:59.062480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38641e1-0768-4870-ac56-d610292281a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:59.063216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38641e1-0768-4870-ac56-d610292281a5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:59.063737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38641e1-0768-4870-ac56-d610292281a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:59.063737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38641e1-0768-4870-ac56-d610292281a5 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:59.064883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38641e1-0768-4870-ac56-d610292281a5 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:59.066349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38641e1-0768-4870-ac56-d610292281a5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:59.068894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 170/356] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:52:58 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:59.069121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f8ad-e6fd-41e4-ae55-5dce8b7c147a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:59.070277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f8ad-e6fd-41e4-ae55-5dce8b7c147a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:59.070277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f8ad-e6fd-41e4-ae55-5dce8b7c147a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:59.074263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 171/357] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:52:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:52:59.074263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7f8ad-e6fd-41e4-ae55-5dce8b7c147a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:59.091554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:59.158929 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d9e0b-b44b-47fc-a430-ff21300ee23e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:59.197070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7f8ad-e6fd-41e4-ae55-5dce8b7c147a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:59.205884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a7750-ed95-4759-8ae9-8749230bf357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:59.206874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a7750-ed95-4759-8ae9-8749230bf357 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:59.207414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a7750-ed95-4759-8ae9-8749230bf357 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:59.207947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a7750-ed95-4759-8ae9-8749230bf357 None admin] PATH_INFO: `/v3/users/b963460aced047ed90ee5ccfbe7bff9c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:59.209376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a7750-ed95-4759-8ae9-8749230bf357 None admin] RBAC: Authorizing `identity:get_user(user_id=b963460aced047ed90ee5ccfbe7bff9c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:59.211069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a7750-ed95-4759-8ae9-8749230bf357 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:59.218406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 187/358] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:52:59 2022] GET /identity/v3/users/b963460aced047ed90ee5ccfbe7bff9c => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:59.219680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a7750-ed95-4759-8ae9-8749230bf357 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:59.234098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:59.240949 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816dcf6b-f47f-4734-b15f-3b8834047b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:59.241567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dcf6b-f47f-4734-b15f-3b8834047b8b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:59.241845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dcf6b-f47f-4734-b15f-3b8834047b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:59.242115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dcf6b-f47f-4734-b15f-3b8834047b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:59.242845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816dcf6b-f47f-4734-b15f-3b8834047b8b None admin] RBAC: Adding query filter params (group.id=45dd57e06f914c9c80d7ab6f1f5e608d, role.id=34ccf81a9f53404e9edefd846827e5b7, scope.project.id=ba7f09585e054b049eec4280fdd8896c) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:59.243196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816dcf6b-f47f-4734-b15f-3b8834047b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:59.244669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816dcf6b-f47f-4734-b15f-3b8834047b8b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:59.254817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 172/359] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 22 16:52:59 2022] GET /identity/v3/role_assignments?group.id=45dd57e06f914c9c80d7ab6f1f5e608d&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:59.255586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816dcf6b-f47f-4734-b15f-3b8834047b8b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:59.368517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:59.372727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a7750-ed95-4759-8ae9-8749230bf357 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:59.411242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816dcf6b-f47f-4734-b15f-3b8834047b8b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:59.426554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081ee6d-fc7e-4547-852e-55a8628e9b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:59.428011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ee6d-fc7e-4547-852e-55a8628e9b84 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:59.428650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ee6d-fc7e-4547-852e-55a8628e9b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:59.428932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ee6d-fc7e-4547-852e-55a8628e9b84 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:59.429644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081ee6d-fc7e-4547-852e-55a8628e9b84 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:59.429947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081ee6d-fc7e-4547-852e-55a8628e9b84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:59.431389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081ee6d-fc7e-4547-852e-55a8628e9b84 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:59.446997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 188/360] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:52:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:59.446997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d39785-351f-49e2-a737-069b30a37843 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:59.446997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d39785-351f-49e2-a737-069b30a37843 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:59.446997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d39785-351f-49e2-a737-069b30a37843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:59.458093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93b39f-e09c-47bb-ac1a-54e0a7f0038d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:59.458093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93b39f-e09c-47bb-ac1a-54e0a7f0038d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:59.458093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93b39f-e09c-47bb-ac1a-54e0a7f0038d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:59.458093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93b39f-e09c-47bb-ac1a-54e0a7f0038d None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:59.458774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93b39f-e09c-47bb-ac1a-54e0a7f0038d None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:59.460246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93b39f-e09c-47bb-ac1a-54e0a7f0038d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:59.464022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 173/361] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:52:59 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:59.464833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93b39f-e09c-47bb-ac1a-54e0a7f0038d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:59.474095 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-25d39785-351f-49e2-a737-069b30a37843 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:52:59.475344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-25d39785-351f-49e2-a737-069b30a37843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:52:59.562242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 189/362] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:52:59.563208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d39785-351f-49e2-a737-069b30a37843 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:59.566634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:59.598557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93b39f-e09c-47bb-ac1a-54e0a7f0038d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:59.638887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebfbce-3999-4596-a6f6-93c8426152f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:59.639550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebfbce-3999-4596-a6f6-93c8426152f5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:59.639838 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebfbce-3999-4596-a6f6-93c8426152f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:59.640110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebfbce-3999-4596-a6f6-93c8426152f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:59.640844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebfbce-3999-4596-a6f6-93c8426152f5 None admin] RBAC: Adding query filter params (role.id=0ab8ef19a25b4e489c9abb312698599c, user.id=b963460aced047ed90ee5ccfbe7bff9c, scope.system=all) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:59.641152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebfbce-3999-4596-a6f6-93c8426152f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:59.642588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebfbce-3999-4596-a6f6-93c8426152f5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:59.655983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 174/363] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 22 16:52:59 2022] GET /identity/v3/role_assignments?user.id=b963460aced047ed90ee5ccfbe7bff9c&scope.system=all&role.id=0ab8ef19a25b4e489c9abb312698599c => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:59.660273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebfbce-3999-4596-a6f6-93c8426152f5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:59.667239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:59.698216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d39785-351f-49e2-a737-069b30a37843 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:59.742775 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e8121-bfc9-4441-b5fa-e7187d73be24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:59.743458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e8121-bfc9-4441-b5fa-e7187d73be24 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:59.744134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e8121-bfc9-4441-b5fa-e7187d73be24 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:59.744556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e8121-bfc9-4441-b5fa-e7187d73be24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:59.764897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e8121-bfc9-4441-b5fa-e7187d73be24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-233e8121-bfc9-4441-b5fa-e7187d73be24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:52:59.770839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:52:59.771867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:52:59.772681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 190/364] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:52:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:52:59.773858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e8121-bfc9-4441-b5fa-e7187d73be24 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:52:59.852605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:52:59.893802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebfbce-3999-4596-a6f6-93c8426152f5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:59.905596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e8121-bfc9-4441-b5fa-e7187d73be24 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:52:59.943074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acc309-f9d0-4334-b579-7574ade24135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:59.943706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acc309-f9d0-4334-b579-7574ade24135 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:59.943986 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acc309-f9d0-4334-b579-7574ade24135 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:59.944318 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acc309-f9d0-4334-b579-7574ade24135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:59.951306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8c81c-1551-4c56-922e-d4d8609d6cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:52:59.951925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8c81c-1551-4c56-922e-d4d8609d6cfd None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:52:59.953116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8c81c-1551-4c56-922e-d4d8609d6cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:52:59.953391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8c81c-1551-4c56-922e-d4d8609d6cfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:52:59.954133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8c81c-1551-4c56-922e-d4d8609d6cfd None admin] RBAC: Adding query filter params (group.id=45dd57e06f914c9c80d7ab6f1f5e608d, role.id=372b98bd7cdc4f7fbd4b812a50380cef, scope.project.id=ba7f09585e054b049eec4280fdd8896c) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:52:59.954435 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8c81c-1551-4c56-922e-d4d8609d6cfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:52:59.955907 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8c81c-1551-4c56-922e-d4d8609d6cfd None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:52:59.963246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 191/365] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 22 16:52:59 2022] GET /identity/v3/role_assignments?group.id=45dd57e06f914c9c80d7ab6f1f5e608d&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=372b98bd7cdc4f7fbd4b812a50380cef => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:52:59.964078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8c81c-1551-4c56-922e-d4d8609d6cfd None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:00.067999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13acc309-f9d0-4334-b579-7574ade24135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:00.069501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13acc309-f9d0-4334-b579-7574ade24135 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:00.072734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:00.117026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8c81c-1551-4c56-922e-d4d8609d6cfd None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:00.153995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:00.168592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0afcf-f110-4312-808a-42d522b59b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:00.169515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0afcf-f110-4312-808a-42d522b59b6e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:00.170287 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0afcf-f110-4312-808a-42d522b59b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:00.170725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0afcf-f110-4312-808a-42d522b59b6e None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:00.172633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0afcf-f110-4312-808a-42d522b59b6e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:00.173121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0afcf-f110-4312-808a-42d522b59b6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:00.174998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0afcf-f110-4312-808a-42d522b59b6e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:00.186121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 192/366] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:52:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:00.187081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0afcf-f110-4312-808a-42d522b59b6e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:00.215780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 175/367] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:00.218810 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acc309-f9d0-4334-b579-7574ade24135 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:00.312761 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0afcf-f110-4312-808a-42d522b59b6e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:00.355690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acc309-f9d0-4334-b579-7574ade24135 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:00.357834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b49f9-3175-468c-9a0b-9851d9b33c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:00.359500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b49f9-3175-468c-9a0b-9851d9b33c2b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:00.359930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b49f9-3175-468c-9a0b-9851d9b33c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:00.360420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b49f9-3175-468c-9a0b-9851d9b33c2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:00.364591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:00.377423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b49f9-3175-468c-9a0b-9851d9b33c2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-282b49f9-3175-468c-9a0b-9851d9b33c2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:00.385145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:00.386096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:00.386882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:00.389243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 193/368] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:00.390438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b49f9-3175-468c-9a0b-9851d9b33c2b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:00.409114 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4526ea5-d188-48a9-be85-c12a68322531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:00.410134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4526ea5-d188-48a9-be85-c12a68322531 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:00.410496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4526ea5-d188-48a9-be85-c12a68322531 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:00.410844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4526ea5-d188-48a9-be85-c12a68322531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:00.452011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:00.508376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4526ea5-d188-48a9-be85-c12a68322531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:00.508690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4526ea5-d188-48a9-be85-c12a68322531 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:00.538198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b49f9-3175-468c-9a0b-9851d9b33c2b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:00.591947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae7819-8cb9-4c61-9fbf-a6ed028114ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:00.592858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae7819-8cb9-4c61-9fbf-a6ed028114ba None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:00.594099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae7819-8cb9-4c61-9fbf-a6ed028114ba None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:00.594375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae7819-8cb9-4c61-9fbf-a6ed028114ba None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:00.595796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae7819-8cb9-4c61-9fbf-a6ed028114ba None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:00.597604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae7819-8cb9-4c61-9fbf-a6ed028114ba None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:00.599713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 194/369] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:00 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:00.603532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae7819-8cb9-4c61-9fbf-a6ed028114ba None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:00.647387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 176/370] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:00.648718 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4526ea5-d188-48a9-be85-c12a68322531 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:00.654572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:00.735291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae7819-8cb9-4c61-9fbf-a6ed028114ba None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:00.757561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:00.778877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4526ea5-d188-48a9-be85-c12a68322531 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:00.796529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adb399-6563-4b09-afc2-45dfa751144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:00.797147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adb399-6563-4b09-afc2-45dfa751144c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:00.797430 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adb399-6563-4b09-afc2-45dfa751144c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:00.797705 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adb399-6563-4b09-afc2-45dfa751144c None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:00.798376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adb399-6563-4b09-afc2-45dfa751144c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:00.798939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adb399-6563-4b09-afc2-45dfa751144c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:00.800343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adb399-6563-4b09-afc2-45dfa751144c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:00.815343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 195/371] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:00.816207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adb399-6563-4b09-afc2-45dfa751144c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:00.826916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f863f6-57eb-47b1-b99a-1a32806f58b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:00.827931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f863f6-57eb-47b1-b99a-1a32806f58b6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:00.828400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f863f6-57eb-47b1-b99a-1a32806f58b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:00.828878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f863f6-57eb-47b1-b99a-1a32806f58b6 None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:00.830291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f863f6-57eb-47b1-b99a-1a32806f58b6 None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:00.832627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f863f6-57eb-47b1-b99a-1a32806f58b6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:00.835324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 177/372] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:00 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:00.836944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f863f6-57eb-47b1-b99a-1a32806f58b6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:00.954311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:00.956631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adb399-6563-4b09-afc2-45dfa751144c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:00.995246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f863f6-57eb-47b1-b99a-1a32806f58b6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:01.002772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e39d09-db73-4083-b499-d0525805b29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:01.003563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e39d09-db73-4083-b499-d0525805b29a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:01.003942 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e39d09-db73-4083-b499-d0525805b29a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:01.004317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e39d09-db73-4083-b499-d0525805b29a None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:01.005575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e39d09-db73-4083-b499-d0525805b29a None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:01.007377 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e39d09-db73-4083-b499-d0525805b29a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:01.009241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 196/373] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:00 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:01.015303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e39d09-db73-4083-b499-d0525805b29a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:01.036767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534df50-cad5-4025-bd93-b60b82372e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:01.037607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534df50-cad5-4025-bd93-b60b82372e7c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:01.037974 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534df50-cad5-4025-bd93-b60b82372e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:01.038346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534df50-cad5-4025-bd93-b60b82372e7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:01.075129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:01.131709 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534df50-cad5-4025-bd93-b60b82372e7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:01.133464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534df50-cad5-4025-bd93-b60b82372e7c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:01.146307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e39d09-db73-4083-b499-d0525805b29a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:01.193443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341ac19-a819-45e3-9e13-c6646fb57709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:01.194176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341ac19-a819-45e3-9e13-c6646fb57709 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:01.194545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341ac19-a819-45e3-9e13-c6646fb57709 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:01.194864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341ac19-a819-45e3-9e13-c6646fb57709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:01.195701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341ac19-a819-45e3-9e13-c6646fb57709 None admin] RBAC: Adding query filter params (group.id=45dd57e06f914c9c80d7ab6f1f5e608d, role.id=34ccf81a9f53404e9edefd846827e5b7, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:01.196081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341ac19-a819-45e3-9e13-c6646fb57709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:01.197594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341ac19-a819-45e3-9e13-c6646fb57709 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:01.205449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 197/374] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 22 16:53:01 2022] GET /identity/v3/role_assignments?group.id=45dd57e06f914c9c80d7ab6f1f5e608d&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:01.208206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341ac19-a819-45e3-9e13-c6646fb57709 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:01.241742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:01.309179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 178/375] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:01.316168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534df50-cad5-4025-bd93-b60b82372e7c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:01.436575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341ac19-a819-45e3-9e13-c6646fb57709 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:01.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:01.489775 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc72dc2-6272-45f4-87dc-11fc5b3aa685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:01.490419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc72dc2-6272-45f4-87dc-11fc5b3aa685 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:01.490702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc72dc2-6272-45f4-87dc-11fc5b3aa685 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:01.491010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc72dc2-6272-45f4-87dc-11fc5b3aa685 None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:01.495202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc72dc2-6272-45f4-87dc-11fc5b3aa685 None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:01.496758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc72dc2-6272-45f4-87dc-11fc5b3aa685 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:01.498457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 198/376] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:01 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:01.505040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc72dc2-6272-45f4-87dc-11fc5b3aa685 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:01.544589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534df50-cad5-4025-bd93-b60b82372e7c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:01.601550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b79466-4674-4034-98ec-bb6d190cc2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:01.602512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b79466-4674-4034-98ec-bb6d190cc2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:01.603030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b79466-4674-4034-98ec-bb6d190cc2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:01.603492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b79466-4674-4034-98ec-bb6d190cc2a8 None admin] PATH_INFO: `/v3/projects/5aa999029c6348189112ae4315aae1bd` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:01.607665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b79466-4674-4034-98ec-bb6d190cc2a8 None admin] RBAC: Authorizing `identity:get_project(project_id=5aa999029c6348189112ae4315aae1bd)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:01.609427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b79466-4674-4034-98ec-bb6d190cc2a8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:01.611366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 179/377] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:01 2022] GET /identity/v3/projects/5aa999029c6348189112ae4315aae1bd => generated 344 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:01.632351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b79466-4674-4034-98ec-bb6d190cc2a8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:01.632747 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc72dc2-6272-45f4-87dc-11fc5b3aa685 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:01.670968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bafc8-faa2-4756-abf5-5aa10c7eed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:01.671316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bafc8-faa2-4756-abf5-5aa10c7eed7b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:01.671516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bafc8-faa2-4756-abf5-5aa10c7eed7b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:01.671785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bafc8-faa2-4756-abf5-5aa10c7eed7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:01.672509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bafc8-faa2-4756-abf5-5aa10c7eed7b None admin] RBAC: Adding query filter params (group.id=45dd57e06f914c9c80d7ab6f1f5e608d, role.id=372b98bd7cdc4f7fbd4b812a50380cef, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:01.672813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bafc8-faa2-4756-abf5-5aa10c7eed7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:01.674528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bafc8-faa2-4756-abf5-5aa10c7eed7b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:01.681235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 199/378] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 22 16:53:01 2022] GET /identity/v3/role_assignments?group.id=45dd57e06f914c9c80d7ab6f1f5e608d&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=372b98bd7cdc4f7fbd4b812a50380cef => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:01.706407 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:01.802122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b79466-4674-4034-98ec-bb6d190cc2a8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:01.856026 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e23b8-0b55-467b-85ab-a9fdd921ee3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:01.856609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e23b8-0b55-467b-85ab-a9fdd921ee3f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:01.856891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e23b8-0b55-467b-85ab-a9fdd921ee3f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:01.857158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e23b8-0b55-467b-85ab-a9fdd921ee3f None admin] PATH_INFO: `/v3/groups/bd309266bd6647faa394c8dcc487fad9` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:01.864107 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e23b8-0b55-467b-85ab-a9fdd921ee3f None admin] RBAC: Authorizing `identity:get_group(group_id=bd309266bd6647faa394c8dcc487fad9)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:01.865602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e23b8-0b55-467b-85ab-a9fdd921ee3f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:01.867690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 180/379] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:01 2022] GET /identity/v3/groups/bd309266bd6647faa394c8dcc487fad9 => generated 229 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:01.874073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bafc8-faa2-4756-abf5-5aa10c7eed7b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:01.907260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fe547-d70f-4cb0-a279-3040cf9ac2ab None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:01.907589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fe547-d70f-4cb0-a279-3040cf9ac2ab None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:01.907811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fe547-d70f-4cb0-a279-3040cf9ac2ab None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:01.909428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 181/380] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:01.915735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732baed-58b9-4465-8292-727be5b9a279 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:01.916016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732baed-58b9-4465-8292-727be5b9a279 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:01.916281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732baed-58b9-4465-8292-727be5b9a279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:01.938279 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d732baed-58b9-4465-8292-727be5b9a279 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:01.939500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d732baed-58b9-4465-8292-727be5b9a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:01.951379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:02.021875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 182/381] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:02.055514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bafc8-faa2-4756-abf5-5aa10c7eed7b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:02.097220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52204d37-96ea-4470-b447-06ea5419b188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:02.097829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52204d37-96ea-4470-b447-06ea5419b188 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:02.098109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52204d37-96ea-4470-b447-06ea5419b188 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:02.098383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52204d37-96ea-4470-b447-06ea5419b188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:02.099159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52204d37-96ea-4470-b447-06ea5419b188 None admin] RBAC: Adding query filter params (group.id=bd309266bd6647faa394c8dcc487fad9, role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=5aa999029c6348189112ae4315aae1bd) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:02.099464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52204d37-96ea-4470-b447-06ea5419b188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:02.100919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52204d37-96ea-4470-b447-06ea5419b188 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:02.110634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 200/382] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 22 16:53:01 2022] GET /identity/v3/role_assignments?group.id=bd309266bd6647faa394c8dcc487fad9&scope.project.id=5aa999029c6348189112ae4315aae1bd&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:02.361406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732baed-58b9-4465-8292-727be5b9a279 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:02.409063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:02.500937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732baed-58b9-4465-8292-727be5b9a279 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:02.537942 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32681a3-7e65-48ec-9338-58ffa66f724c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:02.538677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32681a3-7e65-48ec-9338-58ffa66f724c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:02.538977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32681a3-7e65-48ec-9338-58ffa66f724c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:02.539252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32681a3-7e65-48ec-9338-58ffa66f724c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:02.552493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32681a3-7e65-48ec-9338-58ffa66f724c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-d32681a3-7e65-48ec-9338-58ffa66f724c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:02.557899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:02.559533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:02.560322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 183/383] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:02.565749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52204d37-96ea-4470-b447-06ea5419b188 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:02.676481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52204d37-96ea-4470-b447-06ea5419b188 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:02.684220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:02.695276 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d8eae-5614-49a3-ae22-20e50b700853 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:02.695915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d8eae-5614-49a3-ae22-20e50b700853 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:02.696406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d8eae-5614-49a3-ae22-20e50b700853 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:02.698290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 184/384] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:02.704500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40ed0-dd5d-44e1-b9b0-c0ce04161050 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:02.705175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40ed0-dd5d-44e1-b9b0-c0ce04161050 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:02.705748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40ed0-dd5d-44e1-b9b0-c0ce04161050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:02.717634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba2159-a676-4171-9d76-fd3a0a6483ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:02.718230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2159-a676-4171-9d76-fd3a0a6483ae None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:02.718510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2159-a676-4171-9d76-fd3a0a6483ae None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:02.718782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2159-a676-4171-9d76-fd3a0a6483ae None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:02.719499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba2159-a676-4171-9d76-fd3a0a6483ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:02.719802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba2159-a676-4171-9d76-fd3a0a6483ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:02.721286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba2159-a676-4171-9d76-fd3a0a6483ae None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:02.732914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 201/385] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:53:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:02.734059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-58d40ed0-dd5d-44e1-b9b0-c0ce04161050 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:02.739197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-58d40ed0-dd5d-44e1-b9b0-c0ce04161050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:02.741603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba2159-a676-4171-9d76-fd3a0a6483ae None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:02.820375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 185/386] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:02.939468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba2159-a676-4171-9d76-fd3a0a6483ae None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:02.986620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f35335-9361-40b3-9898-e2fa76f403ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:02.987296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f35335-9361-40b3-9898-e2fa76f403ef None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:02.987574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f35335-9361-40b3-9898-e2fa76f403ef None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:02.987845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f35335-9361-40b3-9898-e2fa76f403ef None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:02.991672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f35335-9361-40b3-9898-e2fa76f403ef None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:02.993119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f35335-9361-40b3-9898-e2fa76f403ef None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:02.995152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 202/387] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:02 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:03.000508 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d40ed0-dd5d-44e1-b9b0-c0ce04161050 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:03.044899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:03.158512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d40ed0-dd5d-44e1-b9b0-c0ce04161050 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:03.197170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f35335-9361-40b3-9898-e2fa76f403ef None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:03.205868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aed278-f700-445d-8651-465e173fe1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:03.206518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aed278-f700-445d-8651-465e173fe1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:03.206795 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aed278-f700-445d-8651-465e173fe1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:03.207097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aed278-f700-445d-8651-465e173fe1f8 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:03.208235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aed278-f700-445d-8651-465e173fe1f8 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:03.209740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aed278-f700-445d-8651-465e173fe1f8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:03.215676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 186/388] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:02 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:03.217901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aed278-f700-445d-8651-465e173fe1f8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:03.259325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:03.390166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aed278-f700-445d-8651-465e173fe1f8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:03.408893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f35335-9361-40b3-9898-e2fa76f403ef None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:03.411813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:03.431880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057365c6-9717-4b43-8513-8bae9d9221dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:03.432560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057365c6-9717-4b43-8513-8bae9d9221dc None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:03.432842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057365c6-9717-4b43-8513-8bae9d9221dc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:03.433116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057365c6-9717-4b43-8513-8bae9d9221dc None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c/groups/45dd57e06f914c9c80d7ab6f1f5e608d/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:03.437357 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057365c6-9717-4b43-8513-8bae9d9221dc None admin] RBAC: Authorizing `identity:create_grant(project_id=ba7f09585e054b049eec4280fdd8896c, group_id=45dd57e06f914c9c80d7ab6f1f5e608d, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:03.438834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057365c6-9717-4b43-8513-8bae9d9221dc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:03.459965 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 187/389] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 22 16:53:03 2022] PUT /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c/groups/45dd57e06f914c9c80d7ab6f1f5e608d/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 16:53:03.477524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee021d-83fe-45e5-bc88-5398f8f54220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:03.478277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee021d-83fe-45e5-bc88-5398f8f54220 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:03.478567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee021d-83fe-45e5-bc88-5398f8f54220 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:03.478841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee021d-83fe-45e5-bc88-5398f8f54220 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:03.493651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee021d-83fe-45e5-bc88-5398f8f54220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-24ee021d-83fe-45e5-bc88-5398f8f54220 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:03.499508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:03.500482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:03.501479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 203/390] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:53:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:03.508097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057365c6-9717-4b43-8513-8bae9d9221dc None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:03.662626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057365c6-9717-4b43-8513-8bae9d9221dc None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:03.704852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f43778-2d75-499a-86cc-87a563092cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:03.705531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f43778-2d75-499a-86cc-87a563092cac None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:03.705807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f43778-2d75-499a-86cc-87a563092cac None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:03.706125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f43778-2d75-499a-86cc-87a563092cac None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:03.706802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f43778-2d75-499a-86cc-87a563092cac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:03.707159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f43778-2d75-499a-86cc-87a563092cac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:03.708593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f43778-2d75-499a-86cc-87a563092cac None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:03.719937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 188/391] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:03.725687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee021d-83fe-45e5-bc88-5398f8f54220 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:03.847988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:03.853793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f3043-cf0b-4c94-96f7-2a2bc2d31fa2 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:03.854168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee021d-83fe-45e5-bc88-5398f8f54220 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:03.854428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f3043-cf0b-4c94-96f7-2a2bc2d31fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:03.854864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f3043-cf0b-4c94-96f7-2a2bc2d31fa2 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:03.856797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 189/392] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:03.862464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3887a-3f4e-4aec-9ca0-a6b6f514774b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:03.863065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3887a-3f4e-4aec-9ca0-a6b6f514774b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:03.863532 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3887a-3f4e-4aec-9ca0-a6b6f514774b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:03.899649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-14a3887a-3f4e-4aec-9ca0-a6b6f514774b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:03.901991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-14a3887a-3f4e-4aec-9ca0-a6b6f514774b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:03.913304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400e906-e9ba-4161-a028-c599456752cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:03.914249 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400e906-e9ba-4161-a028-c599456752cf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:03.914719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400e906-e9ba-4161-a028-c599456752cf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:03.915221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400e906-e9ba-4161-a028-c599456752cf None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:03.916541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400e906-e9ba-4161-a028-c599456752cf None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:03.918154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400e906-e9ba-4161-a028-c599456752cf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:03.920038 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 204/393] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:03 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:03.925424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:03.927999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400e906-e9ba-4161-a028-c599456752cf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:04.032410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 190/394] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:04.097017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400e906-e9ba-4161-a028-c599456752cf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:04.148755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fb582-6686-4d12-aa25-7b9f0648861f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:04.149404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fb582-6686-4d12-aa25-7b9f0648861f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:04.149687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fb582-6686-4d12-aa25-7b9f0648861f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:04.149960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fb582-6686-4d12-aa25-7b9f0648861f None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:04.151799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fb582-6686-4d12-aa25-7b9f0648861f None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:04.153310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fb582-6686-4d12-aa25-7b9f0648861f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:04.155140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 205/395] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:03 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:04.161126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3887a-3f4e-4aec-9ca0-a6b6f514774b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:04.318778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3887a-3f4e-4aec-9ca0-a6b6f514774b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:04.369718 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326f8f7-d661-4dca-9c2d-ac742ea798e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:04.370759 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326f8f7-d661-4dca-9c2d-ac742ea798e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:04.371334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326f8f7-d661-4dca-9c2d-ac742ea798e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:04.371955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326f8f7-d661-4dca-9c2d-ac742ea798e4 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c/groups/45dd57e06f914c9c80d7ab6f1f5e608d/roles/372b98bd7cdc4f7fbd4b812a50380cef` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:04.373000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fb582-6686-4d12-aa25-7b9f0648861f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:04.374361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326f8f7-d661-4dca-9c2d-ac742ea798e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba7f09585e054b049eec4280fdd8896c, group_id=45dd57e06f914c9c80d7ab6f1f5e608d, role_id=372b98bd7cdc4f7fbd4b812a50380cef)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:04.375845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326f8f7-d661-4dca-9c2d-ac742ea798e4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:04.398472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 191/396] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 22 16:53:04 2022] PUT /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c/groups/45dd57e06f914c9c80d7ab6f1f5e608d/roles/372b98bd7cdc4f7fbd4b812a50380cef => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:53:04.420999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:04.514247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fb582-6686-4d12-aa25-7b9f0648861f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:04.577228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcc4c4-2b47-4c7d-bf1c-58a08dd7076f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:04.578396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcc4c4-2b47-4c7d-bf1c-58a08dd7076f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:04.578965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcc4c4-2b47-4c7d-bf1c-58a08dd7076f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:04.579530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcc4c4-2b47-4c7d-bf1c-58a08dd7076f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:04.600603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcc4c4-2b47-4c7d-bf1c-58a08dd7076f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-d9bcc4c4-2b47-4c7d-bf1c-58a08dd7076f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:04.607975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:04.609222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:53:04.610204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:04.613616 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 206/397] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:04.619466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326f8f7-d661-4dca-9c2d-ac742ea798e4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:04.632693 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:04.776244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326f8f7-d661-4dca-9c2d-ac742ea798e4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:04.786972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a63a3-9b0a-4241-a56b-48f9ad95af2c None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:04.787254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a63a3-9b0a-4241-a56b-48f9ad95af2c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:04.787515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a63a3-9b0a-4241-a56b-48f9ad95af2c None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:04.788842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 207/398] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:53:04.794088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c2a11-2cd0-42e1-a437-efda597f02df None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:04.794369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c2a11-2cd0-42e1-a437-efda597f02df None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:04.794643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c2a11-2cd0-42e1-a437-efda597f02df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:04.817799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5921b1-6e99-4a88-867b-4a75897ef268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:04.818415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5921b1-6e99-4a88-867b-4a75897ef268 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:04.818984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5921b1-6e99-4a88-867b-4a75897ef268 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:04.819264 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5921b1-6e99-4a88-867b-4a75897ef268 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:04.819937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5921b1-6e99-4a88-867b-4a75897ef268 None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:04.820238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5921b1-6e99-4a88-867b-4a75897ef268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:04.821749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5921b1-6e99-4a88-867b-4a75897ef268 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:04.830788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 192/399] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:53:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:04.831893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-780c2a11-2cd0-42e1-a437-efda597f02df None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:04.833104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-780c2a11-2cd0-42e1-a437-efda597f02df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:04.840641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5921b1-6e99-4a88-867b-4a75897ef268 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:04.946309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 208/400] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:04.961057 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:05.010152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5921b1-6e99-4a88-867b-4a75897ef268 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:05.046228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a26ca-79db-4aa4-9010-c3c262110be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:05.047192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a26ca-79db-4aa4-9010-c3c262110be4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:05.047677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a26ca-79db-4aa4-9010-c3c262110be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:05.048123 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a26ca-79db-4aa4-9010-c3c262110be4 None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:05.049741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a26ca-79db-4aa4-9010-c3c262110be4 None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:05.051390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a26ca-79db-4aa4-9010-c3c262110be4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:05.053474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 193/401] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:04 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:05.058828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c2a11-2cd0-42e1-a437-efda597f02df None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:05.089682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:05.180242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c2a11-2cd0-42e1-a437-efda597f02df None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:05.222442 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bb16c-f8dd-4f64-a4ca-5509d3bd100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:05.223116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bb16c-f8dd-4f64-a4ca-5509d3bd100b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:05.223398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bb16c-f8dd-4f64-a4ca-5509d3bd100b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:05.223674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bb16c-f8dd-4f64-a4ca-5509d3bd100b None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:05.224823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bb16c-f8dd-4f64-a4ca-5509d3bd100b None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:05.226258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bb16c-f8dd-4f64-a4ca-5509d3bd100b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:05.228227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 209/402] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:05 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:05.234255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a26ca-79db-4aa4-9010-c3c262110be4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:05.284071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc5ccc-e28c-4e2e-bbbb-3339d9da04a9 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:05.284375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc5ccc-e28c-4e2e-bbbb-3339d9da04a9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:05.284932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc5ccc-e28c-4e2e-bbbb-3339d9da04a9 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:05.286534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 210/403] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:53:05.290572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc5ccc-e28c-4e2e-bbbb-3339d9da04a9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:05.380960 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a26ca-79db-4aa4-9010-c3c262110be4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:05.392632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:05.405955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc5ccc-e28c-4e2e-bbbb-3339d9da04a9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:05.432341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6e9d5-b8d0-461f-a3a0-a7c755ff1a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:05.433490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6e9d5-b8d0-461f-a3a0-a7c755ff1a24 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:05.434685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6e9d5-b8d0-461f-a3a0-a7c755ff1a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:05.435735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6e9d5-b8d0-461f-a3a0-a7c755ff1a24 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/groups/45dd57e06f914c9c80d7ab6f1f5e608d/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:05.438811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6e9d5-b8d0-461f-a3a0-a7c755ff1a24 None admin] RBAC: Authorizing `identity:create_grant(project_id=c24becb4b27f43ec9eeb7b0ed1066542, group_id=45dd57e06f914c9c80d7ab6f1f5e608d, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:05.440530 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6e9d5-b8d0-461f-a3a0-a7c755ff1a24 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:05.443160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:05.449273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d13319-3e60-43a2-b110-a0a7742dc6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:05.450068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d13319-3e60-43a2-b110-a0a7742dc6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:05.450402 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d13319-3e60-43a2-b110-a0a7742dc6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:05.450705 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d13319-3e60-43a2-b110-a0a7742dc6c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:05.463321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 194/404] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 22 16:53:05 2022] PUT /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/groups/45dd57e06f914c9c80d7ab6f1f5e608d/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:53:05.464387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d13319-3e60-43a2-b110-a0a7742dc6c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:05.466007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49311-e7ba-4930-9eee-1c0524e884d9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:05.466546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49311-e7ba-4930-9eee-1c0524e884d9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:05.467042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49311-e7ba-4930-9eee-1c0524e884d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a6d13319-3e60-43a2-b110-a0a7742dc6c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:05.469976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:05.471003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:05.471839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 211/405] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:53:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:05.481444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d13319-3e60-43a2-b110-a0a7742dc6c5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:05.516082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-24e49311-e7ba-4930-9eee-1c0524e884d9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:05.518909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-24e49311-e7ba-4930-9eee-1c0524e884d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:05.678291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 195/406] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:05.709046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d13319-3e60-43a2-b110-a0a7742dc6c5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:05.754979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866263e-83c6-4aab-aef8-a09fa9d26925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:05.755178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866263e-83c6-4aab-aef8-a09fa9d26925 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:05.761158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866263e-83c6-4aab-aef8-a09fa9d26925 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:05.761158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866263e-83c6-4aab-aef8-a09fa9d26925 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:05.761158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866263e-83c6-4aab-aef8-a09fa9d26925 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:05.761158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866263e-83c6-4aab-aef8-a09fa9d26925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:05.761158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866263e-83c6-4aab-aef8-a09fa9d26925 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:05.773409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e49311-e7ba-4930-9eee-1c0524e884d9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:05.775347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 212/407] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:05.926637 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e49311-e7ba-4930-9eee-1c0524e884d9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:05.961369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:05.979686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20bef4-4519-48ea-b03a-dd96e73df342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:05.980334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20bef4-4519-48ea-b03a-dd96e73df342 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:05.981122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20bef4-4519-48ea-b03a-dd96e73df342 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:05.981410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20bef4-4519-48ea-b03a-dd96e73df342 None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:05.982586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20bef4-4519-48ea-b03a-dd96e73df342 None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:05.984046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20bef4-4519-48ea-b03a-dd96e73df342 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:05.985958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 196/408] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:05 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:05.991497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866263e-83c6-4aab-aef8-a09fa9d26925 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:06.015889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:06.023600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20bef4-4519-48ea-b03a-dd96e73df342 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:06.134222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866263e-83c6-4aab-aef8-a09fa9d26925 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:06.169402 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a98c82-e0ac-48b7-a8fe-afc0d8c2ff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:06.170030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a98c82-e0ac-48b7-a8fe-afc0d8c2ff30 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:06.170306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a98c82-e0ac-48b7-a8fe-afc0d8c2ff30 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:06.170746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a98c82-e0ac-48b7-a8fe-afc0d8c2ff30 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:06.171929 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a98c82-e0ac-48b7-a8fe-afc0d8c2ff30 None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:06.173372 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a98c82-e0ac-48b7-a8fe-afc0d8c2ff30 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:06.179057 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 213/409] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:05 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:06.181224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a98c82-e0ac-48b7-a8fe-afc0d8c2ff30 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:06.259021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20bef4-4519-48ea-b03a-dd96e73df342 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:06.286684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:06.298915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07dc3a8-f35f-4906-acbd-c7496b7b1c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:06.299955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dc3a8-f35f-4906-acbd-c7496b7b1c32 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:06.300440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dc3a8-f35f-4906-acbd-c7496b7b1c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:06.300946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dc3a8-f35f-4906-acbd-c7496b7b1c32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:06.306981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a98c82-e0ac-48b7-a8fe-afc0d8c2ff30 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:06.312938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07dc3a8-f35f-4906-acbd-c7496b7b1c32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-e07dc3a8-f35f-4906-acbd-c7496b7b1c32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:06.318344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:06.319490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:06.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:06.327953 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 197/410] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:06.329941 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5b016-aef4-45c5-8daf-5b094ae6f5cf None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:06.330512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5b016-aef4-45c5-8daf-5b094ae6f5cf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:06.331012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5b016-aef4-45c5-8daf-5b094ae6f5cf None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:06.332791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 198/411] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:06.333773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5b016-aef4-45c5-8daf-5b094ae6f5cf None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:06.342660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced58f61-e1ca-4c89-add6-9217fa8ffa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:06.344227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced58f61-e1ca-4c89-add6-9217fa8ffa84 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:06.344510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced58f61-e1ca-4c89-add6-9217fa8ffa84 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:06.344778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced58f61-e1ca-4c89-add6-9217fa8ffa84 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/groups/45dd57e06f914c9c80d7ab6f1f5e608d/roles/372b98bd7cdc4f7fbd4b812a50380cef` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:06.346993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced58f61-e1ca-4c89-add6-9217fa8ffa84 None admin] RBAC: Authorizing `identity:create_grant(project_id=c24becb4b27f43ec9eeb7b0ed1066542, group_id=45dd57e06f914c9c80d7ab6f1f5e608d, role_id=372b98bd7cdc4f7fbd4b812a50380cef)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:06.348420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced58f61-e1ca-4c89-add6-9217fa8ffa84 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:06.367058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 214/412] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 22 16:53:06 2022] PUT /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542/groups/45dd57e06f914c9c80d7ab6f1f5e608d/roles/372b98bd7cdc4f7fbd4b812a50380cef => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 16:53:06.368773 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a035-17ac-4214-9916-6000548c403f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:06.369316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a035-17ac-4214-9916-6000548c403f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:06.369822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a035-17ac-4214-9916-6000548c403f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:06.413320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7b42a035-17ac-4214-9916-6000548c403f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:06.416856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7b42a035-17ac-4214-9916-6000548c403f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:06.445839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:06.510538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5b016-aef4-45c5-8daf-5b094ae6f5cf None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:06.583776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 215/413] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:06.584438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9308c10-f31b-4ce0-b111-a23e0c5a9136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:06.585066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9308c10-f31b-4ce0-b111-a23e0c5a9136 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:06.585346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9308c10-f31b-4ce0-b111-a23e0c5a9136 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:06.585616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9308c10-f31b-4ce0-b111-a23e0c5a9136 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:06.586325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9308c10-f31b-4ce0-b111-a23e0c5a9136 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:06.586582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9308c10-f31b-4ce0-b111-a23e0c5a9136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:06.592126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9308c10-f31b-4ce0-b111-a23e0c5a9136 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:06.602399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 199/414] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:06.608829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42a035-17ac-4214-9916-6000548c403f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:06.683611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:06.717752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42a035-17ac-4214-9916-6000548c403f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:06.762993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0f6f6-9503-487c-9123-0a97c13086f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:06.767060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0f6f6-9503-487c-9123-0a97c13086f3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:06.767060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0f6f6-9503-487c-9123-0a97c13086f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:06.767060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0f6f6-9503-487c-9123-0a97c13086f3 None admin] PATH_INFO: `/v3/groups/bd309266bd6647faa394c8dcc487fad9` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:06.767678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0f6f6-9503-487c-9123-0a97c13086f3 None admin] RBAC: Authorizing `identity:get_group(group_id=bd309266bd6647faa394c8dcc487fad9)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:06.769115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0f6f6-9503-487c-9123-0a97c13086f3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:06.770852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 216/415] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:06 2022] GET /identity/v3/groups/bd309266bd6647faa394c8dcc487fad9 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:06.776520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9308c10-f31b-4ce0-b111-a23e0c5a9136 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:06.919974 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9308c10-f31b-4ce0-b111-a23e0c5a9136 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:06.940116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:06.987689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254f363-14eb-4eda-b54e-8e929e8c75dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:06.988346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254f363-14eb-4eda-b54e-8e929e8c75dc None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:06.988915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254f363-14eb-4eda-b54e-8e929e8c75dc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:06.989197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254f363-14eb-4eda-b54e-8e929e8c75dc None admin] PATH_INFO: `/v3/projects/5aa999029c6348189112ae4315aae1bd` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:06.990373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254f363-14eb-4eda-b54e-8e929e8c75dc None admin] RBAC: Authorizing `identity:get_project(project_id=5aa999029c6348189112ae4315aae1bd)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:06.991922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254f363-14eb-4eda-b54e-8e929e8c75dc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:06.993560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 200/416] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:06 2022] GET /identity/v3/projects/5aa999029c6348189112ae4315aae1bd => generated 344 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:07.000042 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0f6f6-9503-487c-9123-0a97c13086f3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:07.012496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5e564-7292-40fd-964f-fc725508ec21 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:07.012790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5e564-7292-40fd-964f-fc725508ec21 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:07.013116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5e564-7292-40fd-964f-fc725508ec21 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:07.014472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 201/417] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:07.028552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8552784-cb64-4180-a2c0-b4db7fbe6900 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:07.028858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8552784-cb64-4180-a2c0-b4db7fbe6900 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:07.029108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8552784-cb64-4180-a2c0-b4db7fbe6900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:07.064922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f8552784-cb64-4180-a2c0-b4db7fbe6900 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:07.066387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f8552784-cb64-4180-a2c0-b4db7fbe6900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:07.135285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0f6f6-9503-487c-9123-0a97c13086f3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:07.155196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 202/418] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:07.155878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8552784-cb64-4180-a2c0-b4db7fbe6900 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:07.204727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d3eb2-ccbf-4676-b2bd-0ded326bd010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:07.205395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d3eb2-ccbf-4676-b2bd-0ded326bd010 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:07.205674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d3eb2-ccbf-4676-b2bd-0ded326bd010 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:07.205946 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d3eb2-ccbf-4676-b2bd-0ded326bd010 None admin] PATH_INFO: `/v3/projects/5aa999029c6348189112ae4315aae1bd/groups/bd309266bd6647faa394c8dcc487fad9/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:07.208667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d3eb2-ccbf-4676-b2bd-0ded326bd010 None admin] RBAC: Authorizing `identity:create_grant(project_id=5aa999029c6348189112ae4315aae1bd, group_id=bd309266bd6647faa394c8dcc487fad9, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:07.211515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d3eb2-ccbf-4676-b2bd-0ded326bd010 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:07.227322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:07.235518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 217/419] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 22 16:53:06 2022] PUT /identity/v3/projects/5aa999029c6348189112ae4315aae1bd/groups/bd309266bd6647faa394c8dcc487fad9/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:53:07.334265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8552784-cb64-4180-a2c0-b4db7fbe6900 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:07.349632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:07.373610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ae412-b205-499b-ac87-6e1330ebd49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:07.374264 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ae412-b205-499b-ac87-6e1330ebd49f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:07.374546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ae412-b205-499b-ac87-6e1330ebd49f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:07.374820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ae412-b205-499b-ac87-6e1330ebd49f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:07.387599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ae412-b205-499b-ac87-6e1330ebd49f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f66ae412-b205-499b-ac87-6e1330ebd49f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:07.392608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:07.393501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:07.394283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 203/420] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:07.399876 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d3eb2-ccbf-4676-b2bd-0ded326bd010 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:07.534918 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d3eb2-ccbf-4676-b2bd-0ded326bd010 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:07.540149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ae412-b205-499b-ac87-6e1330ebd49f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:07.580524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ea58-c270-49db-b193-072032845318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:07.581140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ea58-c270-49db-b193-072032845318 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:07.581421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ea58-c270-49db-b193-072032845318 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:07.581693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ea58-c270-49db-b193-072032845318 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:07.582361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533ea58-c270-49db-b193-072032845318 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:07.582663 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533ea58-c270-49db-b193-072032845318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:07.584079 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533ea58-c270-49db-b193-072032845318 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:07.594416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 218/421] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:53:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:07.599867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ea58-c270-49db-b193-072032845318 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:07.666223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ae412-b205-499b-ac87-6e1330ebd49f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:07.668266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:07.715969 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ea58-c270-49db-b193-072032845318 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:07.717747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e121a-af2c-426a-8c21-9a2ae3a0a9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:07.718566 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e121a-af2c-426a-8c21-9a2ae3a0a9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:07.719002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e121a-af2c-426a-8c21-9a2ae3a0a9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:07.719413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e121a-af2c-426a-8c21-9a2ae3a0a9b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:07.739895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e121a-af2c-426a-8c21-9a2ae3a0a9b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-451e121a-af2c-426a-8c21-9a2ae3a0a9b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:07.755077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:07.756079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:53:07.756930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:07.757544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 204/422] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:53:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:07.763325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e121a-af2c-426a-8c21-9a2ae3a0a9b7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:07.771948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad73abf-1b46-4ae8-a4e1-0177ec982a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:07.772838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73abf-1b46-4ae8-a4e1-0177ec982a55 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:07.778604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73abf-1b46-4ae8-a4e1-0177ec982a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:07.779226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73abf-1b46-4ae8-a4e1-0177ec982a55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:07.830299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:07.856595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad73abf-1b46-4ae8-a4e1-0177ec982a55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:07.858436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad73abf-1b46-4ae8-a4e1-0177ec982a55 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:07.973801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:07.974018 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e121a-af2c-426a-8c21-9a2ae3a0a9b7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:08.035884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd30ed-083c-44b4-8d53-9a2e44587ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:08.036506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd30ed-083c-44b4-8d53-9a2e44587ddc None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:08.036786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd30ed-083c-44b4-8d53-9a2e44587ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:08.037058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd30ed-083c-44b4-8d53-9a2e44587ddc None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:08.038113 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd30ed-083c-44b4-8d53-9a2e44587ddc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:08.038424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd30ed-083c-44b4-8d53-9a2e44587ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:08.039833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd30ed-083c-44b4-8d53-9a2e44587ddc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:08.040488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 219/423] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:08.041966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e346e-bbaa-41df-8665-ee25af805b8e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:08.042329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e346e-bbaa-41df-8665-ee25af805b8e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:08.042603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e346e-bbaa-41df-8665-ee25af805b8e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:08.049042 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 220/424] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:08.049678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 205/425] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:08.049995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e346e-bbaa-41df-8665-ee25af805b8e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:08.051978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309d0a8-9a30-493b-a592-5f4a727784e7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:08.052267 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309d0a8-9a30-493b-a592-5f4a727784e7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:08.052546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309d0a8-9a30-493b-a592-5f4a727784e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:08.074034 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d309d0a8-9a30-493b-a592-5f4a727784e7 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:08.076540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d309d0a8-9a30-493b-a592-5f4a727784e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:08.167203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 206/426] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:08.168613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309d0a8-9a30-493b-a592-5f4a727784e7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:08.180520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e346e-bbaa-41df-8665-ee25af805b8e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:08.220381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:08.228054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be49c6e-6b1c-4f75-83dc-00d24318cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:08.228695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be49c6e-6b1c-4f75-83dc-00d24318cc74 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:08.229032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be49c6e-6b1c-4f75-83dc-00d24318cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:08.229303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be49c6e-6b1c-4f75-83dc-00d24318cc74 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:08.230478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be49c6e-6b1c-4f75-83dc-00d24318cc74 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:08.233000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be49c6e-6b1c-4f75-83dc-00d24318cc74 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:08.235460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 221/427] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:08 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:08.244403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be49c6e-6b1c-4f75-83dc-00d24318cc74 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:08.303752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:08.401198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309d0a8-9a30-493b-a592-5f4a727784e7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:08.418565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be49c6e-6b1c-4f75-83dc-00d24318cc74 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:08.470021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175d675-2cbc-40e8-8d41-a3a8bc5cae32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:08.470643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175d675-2cbc-40e8-8d41-a3a8bc5cae32 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:08.470925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175d675-2cbc-40e8-8d41-a3a8bc5cae32 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:08.471222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175d675-2cbc-40e8-8d41-a3a8bc5cae32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:08.504129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54824084-1326-40a8-b9f4-d757c4b4d552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:08.504129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54824084-1326-40a8-b9f4-d757c4b4d552 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:08.504129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54824084-1326-40a8-b9f4-d757c4b4d552 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:08.504513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54824084-1326-40a8-b9f4-d757c4b4d552 None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:08.505569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54824084-1326-40a8-b9f4-d757c4b4d552 None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:08.507034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54824084-1326-40a8-b9f4-d757c4b4d552 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:08.508943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 222/428] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:08 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:08.525420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54824084-1326-40a8-b9f4-d757c4b4d552 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:08.562096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175d675-2cbc-40e8-8d41-a3a8bc5cae32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:08.563597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175d675-2cbc-40e8-8d41-a3a8bc5cae32 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:08.589702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:08.666258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:08.677409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54824084-1326-40a8-b9f4-d757c4b4d552 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:08.678009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 207/429] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:08.678883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175d675-2cbc-40e8-8d41-a3a8bc5cae32 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:08.731839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea24b2-e67c-4a9e-9461-3ea41c432248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:08.732854 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea24b2-e67c-4a9e-9461-3ea41c432248 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:08.733135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea24b2-e67c-4a9e-9461-3ea41c432248 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:08.733409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea24b2-e67c-4a9e-9461-3ea41c432248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:08.734146 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea24b2-e67c-4a9e-9461-3ea41c432248 None admin] RBAC: Adding query filter params (group.id=45dd57e06f914c9c80d7ab6f1f5e608d, role.id=34ccf81a9f53404e9edefd846827e5b7, scope.project.id=ba7f09585e054b049eec4280fdd8896c) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:08.734454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea24b2-e67c-4a9e-9461-3ea41c432248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:08.736506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea24b2-e67c-4a9e-9461-3ea41c432248 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:08.748129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 223/430] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 22 16:53:08 2022] GET /identity/v3/role_assignments?group.id=45dd57e06f914c9c80d7ab6f1f5e608d&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:08.748977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea24b2-e67c-4a9e-9461-3ea41c432248 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:08.851215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175d675-2cbc-40e8-8d41-a3a8bc5cae32 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:08.899511 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea24b2-e67c-4a9e-9461-3ea41c432248 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:08.908187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:08.938202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5049c8b-43a8-40d4-a627-3155c4d50de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:08.939140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5049c8b-43a8-40d4-a627-3155c4d50de0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:08.939595 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5049c8b-43a8-40d4-a627-3155c4d50de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:08.940009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5049c8b-43a8-40d4-a627-3155c4d50de0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:08.942239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6e33a-d0f2-4cd9-b7b3-2ab21e04c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:08.942871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6e33a-d0f2-4cd9-b7b3-2ab21e04c6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:08.943205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6e33a-d0f2-4cd9-b7b3-2ab21e04c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:08.943447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6e33a-d0f2-4cd9-b7b3-2ab21e04c6a3 None admin] PATH_INFO: `/v3/projects/ba7f09585e054b049eec4280fdd8896c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:08.944702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6e33a-d0f2-4cd9-b7b3-2ab21e04c6a3 None admin] RBAC: Authorizing `identity:get_project(project_id=ba7f09585e054b049eec4280fdd8896c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:08.946344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6e33a-d0f2-4cd9-b7b3-2ab21e04c6a3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:08.948825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 224/431] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:08 2022] GET /identity/v3/projects/ba7f09585e054b049eec4280fdd8896c => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:08.953510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6e33a-d0f2-4cd9-b7b3-2ab21e04c6a3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:08.971603 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5049c8b-43a8-40d4-a627-3155c4d50de0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-a5049c8b-43a8-40d4-a627-3155c4d50de0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:08.977430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:08.978365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:08.979690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 208/432] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:08.984837 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5049c8b-43a8-40d4-a627-3155c4d50de0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:08.991346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:09.097288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6e33a-d0f2-4cd9-b7b3-2ab21e04c6a3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:09.147452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5049c8b-43a8-40d4-a627-3155c4d50de0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:09.147908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf2568-3e71-451c-a77c-148aa385f250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:09.148614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf2568-3e71-451c-a77c-148aa385f250 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:09.148890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf2568-3e71-451c-a77c-148aa385f250 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:09.149161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf2568-3e71-451c-a77c-148aa385f250 None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:09.150455 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf2568-3e71-451c-a77c-148aa385f250 None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:09.152028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf2568-3e71-451c-a77c-148aa385f250 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:09.154609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 225/433] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:08 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:09.160003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf2568-3e71-451c-a77c-148aa385f250 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:09.190589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5d831-0c74-4b9b-b98f-b2f75a2eca93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:09.191539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5d831-0c74-4b9b-b98f-b2f75a2eca93 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:09.191992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5d831-0c74-4b9b-b98f-b2f75a2eca93 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:09.192438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5d831-0c74-4b9b-b98f-b2f75a2eca93 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:09.193723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5d831-0c74-4b9b-b98f-b2f75a2eca93 None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:09.194213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5d831-0c74-4b9b-b98f-b2f75a2eca93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:09.195843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5d831-0c74-4b9b-b98f-b2f75a2eca93 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:09.205216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 209/434] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:53:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:09.206721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58738c85-9972-491b-8297-2179889e1e07 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:09.207218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58738c85-9972-491b-8297-2179889e1e07 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:09.207671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58738c85-9972-491b-8297-2179889e1e07 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:09.209379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 210/435] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:09.215945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58738c85-9972-491b-8297-2179889e1e07 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:09.290378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:09.299383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:09.307469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf2568-3e71-451c-a77c-148aa385f250 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:09.353571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cf3ed-1815-4951-8cac-7a1bd79c9f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:09.354283 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cf3ed-1815-4951-8cac-7a1bd79c9f5f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:09.354591 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cf3ed-1815-4951-8cac-7a1bd79c9f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:09.354828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cf3ed-1815-4951-8cac-7a1bd79c9f5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:09.355682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cf3ed-1815-4951-8cac-7a1bd79c9f5f None admin] RBAC: Adding query filter params (group.id=45dd57e06f914c9c80d7ab6f1f5e608d, role.id=372b98bd7cdc4f7fbd4b812a50380cef, scope.project.id=ba7f09585e054b049eec4280fdd8896c) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:09.355988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cf3ed-1815-4951-8cac-7a1bd79c9f5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:09.357913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cf3ed-1815-4951-8cac-7a1bd79c9f5f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:09.363609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58738c85-9972-491b-8297-2179889e1e07 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:09.365240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 226/436] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 22 16:53:09 2022] GET /identity/v3/role_assignments?group.id=45dd57e06f914c9c80d7ab6f1f5e608d&scope.project.id=ba7f09585e054b049eec4280fdd8896c&role.id=372b98bd7cdc4f7fbd4b812a50380cef => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:09.366468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4777dc3-411d-43c8-abef-a6c8b5047d8b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:09.366749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4777dc3-411d-43c8-abef-a6c8b5047d8b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:09.368024 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4777dc3-411d-43c8-abef-a6c8b5047d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:09.390862 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b4777dc3-411d-43c8-abef-a6c8b5047d8b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:09.392195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b4777dc3-411d-43c8-abef-a6c8b5047d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:09.419837 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24245e9d-bfe4-4269-a942-02be9d341b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:09.421808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24245e9d-bfe4-4269-a942-02be9d341b63 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:09.422437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24245e9d-bfe4-4269-a942-02be9d341b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:09.422895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24245e9d-bfe4-4269-a942-02be9d341b63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:09.489650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 227/437] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:09.523771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24245e9d-bfe4-4269-a942-02be9d341b63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:09.525364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24245e9d-bfe4-4269-a942-02be9d341b63 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:09.584125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:09.645250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 211/438] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:09.650123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4777dc3-411d-43c8-abef-a6c8b5047d8b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:09.746348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:09.763642 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4777dc3-411d-43c8-abef-a6c8b5047d8b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:09.765305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24245e9d-bfe4-4269-a942-02be9d341b63 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:09.800391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b78506-fb9b-4a42-a119-4f84bb7646b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:09.801136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78506-fb9b-4a42-a119-4f84bb7646b6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:09.801417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78506-fb9b-4a42-a119-4f84bb7646b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:09.801687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78506-fb9b-4a42-a119-4f84bb7646b6 None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:09.802839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b78506-fb9b-4a42-a119-4f84bb7646b6 None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:09.804302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b78506-fb9b-4a42-a119-4f84bb7646b6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:09.806082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 228/439] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:09 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:09.812879 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b78506-fb9b-4a42-a119-4f84bb7646b6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:09.902235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24245e9d-bfe4-4269-a942-02be9d341b63 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:09.940524 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d012200-be7e-486c-a7cc-ff11644fabb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:09.940731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b78506-fb9b-4a42-a119-4f84bb7646b6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:09.941167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d012200-be7e-486c-a7cc-ff11644fabb1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:09.941443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d012200-be7e-486c-a7cc-ff11644fabb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:09.941712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d012200-be7e-486c-a7cc-ff11644fabb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:09.953808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d012200-be7e-486c-a7cc-ff11644fabb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-1d012200-be7e-486c-a7cc-ff11644fabb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:09.958502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:09.959452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:09.960441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 212/440] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:53:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:09.961236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21b43-e604-4303-ac4b-d4fc15d04659 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:09.961236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21b43-e604-4303-ac4b-d4fc15d04659 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:09.961383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21b43-e604-4303-ac4b-d4fc15d04659 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:09.962649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 213/441] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:09.965699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21b43-e604-4303-ac4b-d4fc15d04659 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:09.974675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:09.979936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a210d-9764-4499-8515-5a1dd1af20e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:09.980615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a210d-9764-4499-8515-5a1dd1af20e5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:09.981407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a210d-9764-4499-8515-5a1dd1af20e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:09.981662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a210d-9764-4499-8515-5a1dd1af20e5 None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:09.982790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a210d-9764-4499-8515-5a1dd1af20e5 None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:09.984276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a210d-9764-4499-8515-5a1dd1af20e5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:09.987166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 229/442] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:09 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:09.987166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23721fb2-4e7c-4c81-aeff-7a2eb34e5c2b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:09.987350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23721fb2-4e7c-4c81-aeff-7a2eb34e5c2b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:09.987617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23721fb2-4e7c-4c81-aeff-7a2eb34e5c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:10.006978 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-23721fb2-4e7c-4c81-aeff-7a2eb34e5c2b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:10.008093 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-23721fb2-4e7c-4c81-aeff-7a2eb34e5c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:10.029682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:10.071373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21b43-e604-4303-ac4b-d4fc15d04659 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:10.077734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 230/443] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:10.078572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23721fb2-4e7c-4c81-aeff-7a2eb34e5c2b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:10.111308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e904f-ad5b-426f-8dff-462efb1f724d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:10.111913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e904f-ad5b-426f-8dff-462efb1f724d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:10.112188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e904f-ad5b-426f-8dff-462efb1f724d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:10.112455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e904f-ad5b-426f-8dff-462efb1f724d None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:10.113106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e904f-ad5b-426f-8dff-462efb1f724d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:10.113402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e904f-ad5b-426f-8dff-462efb1f724d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:10.114773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e904f-ad5b-426f-8dff-462efb1f724d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:10.123151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 214/444] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:10.128654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e904f-ad5b-426f-8dff-462efb1f724d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:10.185948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23721fb2-4e7c-4c81-aeff-7a2eb34e5c2b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:10.200476 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:10.223398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d347743-7e5c-40b1-830c-683bae193989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:10.223998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d347743-7e5c-40b1-830c-683bae193989 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:10.224267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d347743-7e5c-40b1-830c-683bae193989 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:10.224534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d347743-7e5c-40b1-830c-683bae193989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:10.225250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d347743-7e5c-40b1-830c-683bae193989 None admin] RBAC: Adding query filter params (group.id=45dd57e06f914c9c80d7ab6f1f5e608d, role.id=34ccf81a9f53404e9edefd846827e5b7, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:10.225552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d347743-7e5c-40b1-830c-683bae193989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:10.227010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d347743-7e5c-40b1-830c-683bae193989 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:10.242989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 231/445] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 22 16:53:10 2022] GET /identity/v3/role_assignments?group.id=45dd57e06f914c9c80d7ab6f1f5e608d&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=34ccf81a9f53404e9edefd846827e5b7 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:10.243186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e904f-ad5b-426f-8dff-462efb1f724d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:10.281708 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c0c26-6e66-4d9e-9907-5233b4276f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:10.282233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c0c26-6e66-4d9e-9907-5233b4276f6e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:10.282509 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c0c26-6e66-4d9e-9907-5233b4276f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:10.282779 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c0c26-6e66-4d9e-9907-5233b4276f6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:10.347710 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d347743-7e5c-40b1-830c-683bae193989 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:10.361961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c0c26-6e66-4d9e-9907-5233b4276f6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:10.363498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c0c26-6e66-4d9e-9907-5233b4276f6e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:10.374829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:10.441432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:10.473323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d347743-7e5c-40b1-830c-683bae193989 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:10.481782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 215/446] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 16:53:10.487438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c0c26-6e66-4d9e-9907-5233b4276f6e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:10.516688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918f57f-062d-45fb-b8b9-4f09d375c31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:10.517582 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918f57f-062d-45fb-b8b9-4f09d375c31c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:10.517863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918f57f-062d-45fb-b8b9-4f09d375c31c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:10.518135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918f57f-062d-45fb-b8b9-4f09d375c31c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:10.531381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918f57f-062d-45fb-b8b9-4f09d375c31c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-7918f57f-062d-45fb-b8b9-4f09d375c31c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:10.536844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:10.537834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:10.538676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 232/447] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:10.544424 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918f57f-062d-45fb-b8b9-4f09d375c31c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:10.630720 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c0c26-6e66-4d9e-9907-5233b4276f6e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:10.646307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:10.670045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf7de9-055f-4c88-a5cd-51790bbc3ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:10.670679 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf7de9-055f-4c88-a5cd-51790bbc3ffc None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:10.670967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf7de9-055f-4c88-a5cd-51790bbc3ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:10.671260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf7de9-055f-4c88-a5cd-51790bbc3ffc None admin] PATH_INFO: `/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:10.671939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918f57f-062d-45fb-b8b9-4f09d375c31c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:10.672385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf7de9-055f-4c88-a5cd-51790bbc3ffc None admin] RBAC: Authorizing `identity:get_project(project_id=c24becb4b27f43ec9eeb7b0ed1066542)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:10.674087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf7de9-055f-4c88-a5cd-51790bbc3ffc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:10.675799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 216/448] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:10 2022] GET /identity/v3/projects/c24becb4b27f43ec9eeb7b0ed1066542 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:10.681179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf7de9-055f-4c88-a5cd-51790bbc3ffc None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:10.707892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649e4a-a27c-4053-a701-dc934bab5759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:10.708502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649e4a-a27c-4053-a701-dc934bab5759 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:10.708786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649e4a-a27c-4053-a701-dc934bab5759 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:10.709062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649e4a-a27c-4053-a701-dc934bab5759 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:10.709723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649e4a-a27c-4053-a701-dc934bab5759 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:10.710025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649e4a-a27c-4053-a701-dc934bab5759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:10.711421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649e4a-a27c-4053-a701-dc934bab5759 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:10.716660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:10.720138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 233/449] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:10.725595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649e4a-a27c-4053-a701-dc934bab5759 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:10.785285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf7de9-055f-4c88-a5cd-51790bbc3ffc None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:10.821487 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b983e-b698-4c22-b581-52a0a50cc044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:10.822035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b983e-b698-4c22-b581-52a0a50cc044 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:10.822306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b983e-b698-4c22-b581-52a0a50cc044 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:10.822571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b983e-b698-4c22-b581-52a0a50cc044 None admin] PATH_INFO: `/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:10.823700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b983e-b698-4c22-b581-52a0a50cc044 None admin] RBAC: Authorizing `identity:get_group(group_id=45dd57e06f914c9c80d7ab6f1f5e608d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:10.825692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b983e-b698-4c22-b581-52a0a50cc044 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:10.827287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 217/450] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:10 2022] GET /identity/v3/groups/45dd57e06f914c9c80d7ab6f1f5e608d => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:10.830641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649e4a-a27c-4053-a701-dc934bab5759 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:10.833258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b983e-b698-4c22-b581-52a0a50cc044 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:10.863379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00495dd0-9670-42c4-a9c5-a2be980c2555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:10.864002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00495dd0-9670-42c4-a9c5-a2be980c2555 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:10.864277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00495dd0-9670-42c4-a9c5-a2be980c2555 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:10.864541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00495dd0-9670-42c4-a9c5-a2be980c2555 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:10.876744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:10.943899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00495dd0-9670-42c4-a9c5-a2be980c2555 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:10.945364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00495dd0-9670-42c4-a9c5-a2be980c2555 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:10.955685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b983e-b698-4c22-b581-52a0a50cc044 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:10.958308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:10.992235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd0bec-9251-4f09-93ff-1be812d771b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:10.992849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd0bec-9251-4f09-93ff-1be812d771b6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:10.993123 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd0bec-9251-4f09-93ff-1be812d771b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:10.993397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd0bec-9251-4f09-93ff-1be812d771b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:10.994118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd0bec-9251-4f09-93ff-1be812d771b6 None admin] RBAC: Adding query filter params (group.id=45dd57e06f914c9c80d7ab6f1f5e608d, role.id=372b98bd7cdc4f7fbd4b812a50380cef, scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:10.994425 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd0bec-9251-4f09-93ff-1be812d771b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:10.995896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd0bec-9251-4f09-93ff-1be812d771b6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:11.002590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 218/451] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 22 16:53:10 2022] GET /identity/v3/role_assignments?group.id=45dd57e06f914c9c80d7ab6f1f5e608d&scope.project.id=c24becb4b27f43ec9eeb7b0ed1066542&role.id=372b98bd7cdc4f7fbd4b812a50380cef => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:11.060590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 234/452] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:11.066711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd0bec-9251-4f09-93ff-1be812d771b6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:11.186310 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd0bec-9251-4f09-93ff-1be812d771b6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:11.226372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7547c3-f1e8-4083-99be-9bb69554323c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:11.227182 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7547c3-f1e8-4083-99be-9bb69554323c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:11.227547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7547c3-f1e8-4083-99be-9bb69554323c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:11.227947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7547c3-f1e8-4083-99be-9bb69554323c None admin] PATH_INFO: `/v3/projects/5aa999029c6348189112ae4315aae1bd` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:11.229232 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7547c3-f1e8-4083-99be-9bb69554323c None admin] RBAC: Authorizing `identity:get_project(project_id=5aa999029c6348189112ae4315aae1bd)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:11.231697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7547c3-f1e8-4083-99be-9bb69554323c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:11.239219 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 219/453] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:53:11 2022] GET /identity/v3/projects/5aa999029c6348189112ae4315aae1bd => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:11.244974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00495dd0-9670-42c4-a9c5-a2be980c2555 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:11.305791 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:11.367627 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00495dd0-9670-42c4-a9c5-a2be980c2555 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:11.401802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfcad7-0762-4fed-a80c-1b5a1a2f5f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:11.402986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfcad7-0762-4fed-a80c-1b5a1a2f5f4c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:11.403268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfcad7-0762-4fed-a80c-1b5a1a2f5f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:11.403539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfcad7-0762-4fed-a80c-1b5a1a2f5f4c None admin] PATH_INFO: `/v3/groups/bd309266bd6647faa394c8dcc487fad9` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:11.404660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfcad7-0762-4fed-a80c-1b5a1a2f5f4c None admin] RBAC: Authorizing `identity:get_group(group_id=bd309266bd6647faa394c8dcc487fad9)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:11.406096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfcad7-0762-4fed-a80c-1b5a1a2f5f4c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:11.407698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 235/454] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 22 16:53:11 2022] GET /identity/v3/groups/bd309266bd6647faa394c8dcc487fad9 => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:11.413564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7547c3-f1e8-4083-99be-9bb69554323c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:11.450388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:11.513378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7547c3-f1e8-4083-99be-9bb69554323c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:11.544883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a21fb-f4d0-4d54-8652-e6d1f3d9edf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:11.545477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a21fb-f4d0-4d54-8652-e6d1f3d9edf1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:11.545753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a21fb-f4d0-4d54-8652-e6d1f3d9edf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:11.546020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a21fb-f4d0-4d54-8652-e6d1f3d9edf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:11.546729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a21fb-f4d0-4d54-8652-e6d1f3d9edf1 None admin] RBAC: Adding query filter params (group.id=bd309266bd6647faa394c8dcc487fad9, role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=5aa999029c6348189112ae4315aae1bd) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:11.547058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a21fb-f4d0-4d54-8652-e6d1f3d9edf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:11.548741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a21fb-f4d0-4d54-8652-e6d1f3d9edf1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:11.554426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 220/455] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 22 16:53:11 2022] GET /identity/v3/role_assignments?group.id=bd309266bd6647faa394c8dcc487fad9&scope.project.id=5aa999029c6348189112ae4315aae1bd&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:14.436817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa556590-e580-4bc7-95d7-02468077887a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:14.437359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa556590-e580-4bc7-95d7-02468077887a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:14.437359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa556590-e580-4bc7-95d7-02468077887a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:14.439001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 236/456] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:14.445116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bfaea-61b2-43e0-aa6d-391dbdbe90d1 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:14.445994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bfaea-61b2-43e0-aa6d-391dbdbe90d1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:14.446422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bfaea-61b2-43e0-aa6d-391dbdbe90d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:14.477175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2b7bfaea-61b2-43e0-aa6d-391dbdbe90d1 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:14.478781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2b7bfaea-61b2-43e0-aa6d-391dbdbe90d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:14.572755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e4776-0c61-4fd1-80df-d021305655c2 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:14.573298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e4776-0c61-4fd1-80df-d021305655c2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:14.573705 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e4776-0c61-4fd1-80df-d021305655c2 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:14.576088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 237/457] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:14.583481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 221/458] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:14.586583 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539a03-f45b-4992-8c84-3932bf73541a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:14.587159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539a03-f45b-4992-8c84-3932bf73541a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:14.587642 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539a03-f45b-4992-8c84-3932bf73541a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:14.611478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-22539a03-f45b-4992-8c84-3932bf73541a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:14.612993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-22539a03-f45b-4992-8c84-3932bf73541a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:14.634852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bad6b-852b-4b1d-b93d-155538093225 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:14.635438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bad6b-852b-4b1d-b93d-155538093225 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:14.635849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bad6b-852b-4b1d-b93d-155538093225 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:14.641941 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 222/459] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:14.643983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a9909-c110-4bae-863e-ee2a7a85547d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:14.644417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a9909-c110-4bae-863e-ee2a7a85547d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:14.644851 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a9909-c110-4bae-863e-ee2a7a85547d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:14.682042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-880a9909-c110-4bae-863e-ee2a7a85547d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:14.683277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-880a9909-c110-4bae-863e-ee2a7a85547d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:14.721427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 238/460] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:14.732085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:14.809306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 223/461] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:14.920276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22539a03-f45b-4992-8c84-3932bf73541a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:14.974678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:15.057740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22539a03-f45b-4992-8c84-3932bf73541a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:15.088527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a9909-c110-4bae-863e-ee2a7a85547d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:15.103047 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe28155-4fd2-4c67-9935-855bdf31689b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:15.104316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28155-4fd2-4c67-9935-855bdf31689b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.104626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28155-4fd2-4c67-9935-855bdf31689b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.104869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28155-4fd2-4c67-9935-855bdf31689b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.117875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe28155-4fd2-4c67-9935-855bdf31689b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:15.119364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe28155-4fd2-4c67-9935-855bdf31689b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-efe28155-4fd2-4c67-9935-855bdf31689b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:15.138680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:15.139706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:15.141154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:15.141154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:15.141154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:15.141154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 239/462] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:15.148334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe28155-4fd2-4c67-9935-855bdf31689b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:15.240595 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a9909-c110-4bae-863e-ee2a7a85547d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:15.259693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:15.270793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe28155-4fd2-4c67-9935-855bdf31689b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:15.280323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1a85b-be77-4839-baff-7bea306877a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:15.281088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1a85b-be77-4839-baff-7bea306877a2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.281441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1a85b-be77-4839-baff-7bea306877a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.281721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1a85b-be77-4839-baff-7bea306877a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.292481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1a85b-be77-4839-baff-7bea306877a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:15.294005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1a85b-be77-4839-baff-7bea306877a2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:15.307790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31505580-607f-43b2-8f65-79b490c1b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:15.308396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31505580-607f-43b2-8f65-79b490c1b4af None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.309198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31505580-607f-43b2-8f65-79b490c1b4af None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.309466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31505580-607f-43b2-8f65-79b490c1b4af None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.310112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31505580-607f-43b2-8f65-79b490c1b4af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:15.310401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31505580-607f-43b2-8f65-79b490c1b4af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-eaa1a85b-be77-4839-baff-7bea306877a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:15.311736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:15.312580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:15.315745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:15.315745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:15.315745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:15.315942 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31505580-607f-43b2-8f65-79b490c1b4af None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:15.316025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 224/463] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:15.316025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1a85b-be77-4839-baff-7bea306877a2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:15.331404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 240/464] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:15.332770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:15.333119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1db62-4c31-4017-9c52-a46132017e4e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.333397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1db62-4c31-4017-9c52-a46132017e4e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.334062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1db62-4c31-4017-9c52-a46132017e4e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.339110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 241/465] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:15.339110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1db62-4c31-4017-9c52-a46132017e4e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:15.439248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1a85b-be77-4839-baff-7bea306877a2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:15.450285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1db62-4c31-4017-9c52-a46132017e4e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:15.485160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a42a14-a715-4b49-acac-863cf836924a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:15.485790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a42a14-a715-4b49-acac-863cf836924a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.486268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a42a14-a715-4b49-acac-863cf836924a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.486538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a42a14-a715-4b49-acac-863cf836924a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.490380 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51160577-5e51-4a77-a87a-52961674717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:15.491205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51160577-5e51-4a77-a87a-52961674717b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.491306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51160577-5e51-4a77-a87a-52961674717b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.491571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51160577-5e51-4a77-a87a-52961674717b None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.492216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51160577-5e51-4a77-a87a-52961674717b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:15.492519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51160577-5e51-4a77-a87a-52961674717b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:15.494389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51160577-5e51-4a77-a87a-52961674717b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:15.497672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a42a14-a715-4b49-acac-863cf836924a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:15.499073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a42a14-a715-4b49-acac-863cf836924a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:15.512407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 242/466] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:15.512407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51160577-5e51-4a77-a87a-52961674717b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-b0a42a14-a715-4b49-acac-863cf836924a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:15.520354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:15.521427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:15.522293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:15.522293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:15.522293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:15.522293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 225/467] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:15.522689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c4e89-c346-407f-9805-0f8a5349398d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.522978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c4e89-c346-407f-9805-0f8a5349398d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.523251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c4e89-c346-407f-9805-0f8a5349398d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.532829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:15.544733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-504c4e89-c346-407f-9805-0f8a5349398d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:15.545923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-504c4e89-c346-407f-9805-0f8a5349398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:15.617794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:15.634995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 226/468] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:15.634995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c4e89-c346-407f-9805-0f8a5349398d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:15.640104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51160577-5e51-4a77-a87a-52961674717b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:15.679151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b88e6d-29a6-48b4-b050-879435b9126b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:15.679801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b88e6d-29a6-48b4-b050-879435b9126b None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.680084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b88e6d-29a6-48b4-b050-879435b9126b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.680360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b88e6d-29a6-48b4-b050-879435b9126b None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.681068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b88e6d-29a6-48b4-b050-879435b9126b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:15.682685 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b88e6d-29a6-48b4-b050-879435b9126b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:15.710291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 243/469] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:53:15 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:53:15.711462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b88e6d-29a6-48b4-b050-879435b9126b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:15.780213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c4e89-c346-407f-9805-0f8a5349398d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:15.811510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:15.838726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b9279-e022-41ac-b8f5-77810441e780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:15.840308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b9279-e022-41ac-b8f5-77810441e780 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.840828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b9279-e022-41ac-b8f5-77810441e780 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.842070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b9279-e022-41ac-b8f5-77810441e780 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.843359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b9279-e022-41ac-b8f5-77810441e780 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:15.845979 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b9279-e022-41ac-b8f5-77810441e780 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:15.855907 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b88e6d-29a6-48b4-b050-879435b9126b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:15.884270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 227/470] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:53:15 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:53:15.886658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360ea3-187c-415c-b51b-772cee7456df None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.887296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360ea3-187c-415c-b51b-772cee7456df None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.887807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360ea3-187c-415c-b51b-772cee7456df None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.889967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 228/471] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:15.896418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542b0b6-190c-4d2b-a31b-047d786c4a80 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.897375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542b0b6-190c-4d2b-a31b-047d786c4a80 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.897984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542b0b6-190c-4d2b-a31b-047d786c4a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.901466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febf3ca-e1c7-422f-950a-9cf45557c848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:15.902089 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febf3ca-e1c7-422f-950a-9cf45557c848 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:15.903041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febf3ca-e1c7-422f-950a-9cf45557c848 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:15.903274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febf3ca-e1c7-422f-950a-9cf45557c848 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:15.904041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febf3ca-e1c7-422f-950a-9cf45557c848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:15.904346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febf3ca-e1c7-422f-950a-9cf45557c848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:15.905859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febf3ca-e1c7-422f-950a-9cf45557c848 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:15.924912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 244/472] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:15.926563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9542b0b6-190c-4d2b-a31b-047d786c4a80 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:15.928127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9542b0b6-190c-4d2b-a31b-047d786c4a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:15.931349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febf3ca-e1c7-422f-950a-9cf45557c848 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:15.996428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:16.025653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 229/473] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:16.027039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542b0b6-190c-4d2b-a31b-047d786c4a80 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:16.076624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febf3ca-e1c7-422f-950a-9cf45557c848 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:16.116641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:16.127205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9b5ce-c40f-448f-8673-db82bc256cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:16.127823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9b5ce-c40f-448f-8673-db82bc256cac None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:16.128104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9b5ce-c40f-448f-8673-db82bc256cac None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:16.128379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9b5ce-c40f-448f-8673-db82bc256cac None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:16.129042 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9b5ce-c40f-448f-8673-db82bc256cac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:16.130490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9b5ce-c40f-448f-8673-db82bc256cac None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:16.159855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 245/474] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:53:15 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:53:16.166867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542b0b6-190c-4d2b-a31b-047d786c4a80 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:16.209521 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd15d8-371c-4987-b687-a8b065d7f41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:16.210185 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd15d8-371c-4987-b687-a8b065d7f41a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:16.210466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd15d8-371c-4987-b687-a8b065d7f41a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:16.210744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd15d8-371c-4987-b687-a8b065d7f41a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:16.223774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd15d8-371c-4987-b687-a8b065d7f41a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:16.225405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd15d8-371c-4987-b687-a8b065d7f41a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-dabd15d8-371c-4987-b687-a8b065d7f41a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:16.243197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:16.244350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:16.245187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:16.245187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:16.245187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:16.245187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 230/475] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:16.251637 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9b5ce-c40f-448f-8673-db82bc256cac None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:16.368815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd15d8-371c-4987-b687-a8b065d7f41a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:16.376610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9b5ce-c40f-448f-8673-db82bc256cac None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:16.409518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:16.420186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c583c0-50fa-4173-9bb6-b5427fb7e7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:16.420802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c583c0-50fa-4173-9bb6-b5427fb7e7cf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:16.421078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c583c0-50fa-4173-9bb6-b5427fb7e7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:16.421352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c583c0-50fa-4173-9bb6-b5427fb7e7cf None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:16.421997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c583c0-50fa-4173-9bb6-b5427fb7e7cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:16.422293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c583c0-50fa-4173-9bb6-b5427fb7e7cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:16.424097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c583c0-50fa-4173-9bb6-b5427fb7e7cf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:16.438913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 246/476] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:16.444364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c583c0-50fa-4173-9bb6-b5427fb7e7cf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:16.512879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd15d8-371c-4987-b687-a8b065d7f41a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:16.529656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:16.563440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3469ef0-49b5-49d6-9e3e-481e0ce91860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:16.564069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3469ef0-49b5-49d6-9e3e-481e0ce91860 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:16.564336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3469ef0-49b5-49d6-9e3e-481e0ce91860 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:16.564600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3469ef0-49b5-49d6-9e3e-481e0ce91860 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:16.565556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3469ef0-49b5-49d6-9e3e-481e0ce91860 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:16.567661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3469ef0-49b5-49d6-9e3e-481e0ce91860 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:16.573833 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c583c0-50fa-4173-9bb6-b5427fb7e7cf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:16.577517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.sql.core [None req-c3469ef0-49b5-49d6-9e3e-481e0ce91860 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 16:53:16.577517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 16:53:16.577517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [parameters: {'id': '4f6ce92c117f432a96c180ed6bd3a3c7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 16:53:16.577517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81562) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c3469ef0-49b5-49d6-9e3e-481e0ce91860 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application conn.query(q) Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application result.read() Sep 22 16:53:16.611609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return ref.to_dict() Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 16:53:16.612675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application next(self.gen) Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application yield resource Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application next(self.gen) Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise self.value Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application session.commit() Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 16:53:16.613691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self.session.flush() Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application compat.raise_( Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 16:53:16.614622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 16:53:16.615540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application conn.query(q) Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application result.read() Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 16:53:16.616366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [parameters: {'id': '4f6ce92c117f432a96c180ed6bd3a3c7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:16.617262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 16:53:16.618101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:16.621215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 231/477] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 22 16:53:16 2022] POST /identity/v3/roles => generated 144 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:16.633167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3469ef0-49b5-49d6-9e3e-481e0ce91860 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:16.661926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ebfa-13e3-41a8-b311-e45d837ede30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:16.663683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ebfa-13e3-41a8-b311-e45d837ede30 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:16.664135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ebfa-13e3-41a8-b311-e45d837ede30 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:16.665395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ebfa-13e3-41a8-b311-e45d837ede30 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:16.666216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152ebfa-13e3-41a8-b311-e45d837ede30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:16.669148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152ebfa-13e3-41a8-b311-e45d837ede30 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:16.710223 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 247/478] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:53:16 2022] POST /identity/v3/users => generated 249 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 16:53:16.800352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:16.815455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3469ef0-49b5-49d6-9e3e-481e0ce91860 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:16.878425 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8e170-b286-4392-9178-dff5ff01fa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:16.879468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8e170-b286-4392-9178-dff5ff01fa7a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:16.879950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8e170-b286-4392-9178-dff5ff01fa7a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:16.880443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8e170-b286-4392-9178-dff5ff01fa7a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:16.900309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8e170-b286-4392-9178-dff5ff01fa7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-86a8e170-b286-4392-9178-dff5ff01fa7a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:16.906378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:16.907392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 16:53:16.908173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:16.910721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 232/479] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:53:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:16.917240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ebfa-13e3-41a8-b311-e45d837ede30 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:17.037924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:17.044905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ebfa-13e3-41a8-b311-e45d837ede30 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:17.102157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79d3a-9bf9-48e7-aff0-9ffc49149249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:17.102809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79d3a-9bf9-48e7-aff0-9ffc49149249 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:17.103130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79d3a-9bf9-48e7-aff0-9ffc49149249 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:17.103406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79d3a-9bf9-48e7-aff0-9ffc49149249 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:17.104092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79d3a-9bf9-48e7-aff0-9ffc49149249 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:17.104396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79d3a-9bf9-48e7-aff0-9ffc49149249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:17.106375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79d3a-9bf9-48e7-aff0-9ffc49149249 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:17.116274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 248/480] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:18.538250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41341070-02e6-4428-9b60-8404838ade84 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:18.538774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41341070-02e6-4428-9b60-8404838ade84 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:18.538774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41341070-02e6-4428-9b60-8404838ade84 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:18.540418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 233/481] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:18.545794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30389b0-c158-449a-84e5-5d9557147805 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:18.546254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30389b0-c158-449a-84e5-5d9557147805 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:18.546327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30389b0-c158-449a-84e5-5d9557147805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:18.576267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b30389b0-c158-449a-84e5-5d9557147805 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:18.578205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b30389b0-c158-449a-84e5-5d9557147805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:18.669963 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 249/482] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:18.871426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff8d61-855f-468f-949a-c3d98946a967 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:18.871734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff8d61-855f-468f-949a-c3d98946a967 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:18.871902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff8d61-855f-468f-949a-c3d98946a967 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:18.873645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 234/483] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:18.879528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dfef7-1458-48b6-a0eb-03c83da85f12 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:18.879818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dfef7-1458-48b6-a0eb-03c83da85f12 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:18.880101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dfef7-1458-48b6-a0eb-03c83da85f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:18.899284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-137dfef7-1458-48b6-a0eb-03c83da85f12 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:18.900494 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-137dfef7-1458-48b6-a0eb-03c83da85f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:18.980490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 250/484] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:18.994117 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d960f48-8374-4d8c-a933-4bc1b82d9751 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:18.994802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d960f48-8374-4d8c-a933-4bc1b82d9751 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:18.995347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d960f48-8374-4d8c-a933-4bc1b82d9751 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:18.997883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 235/485] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:19.003504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504d3d-4d3c-468f-893b-1a962e693e71 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:19.004013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504d3d-4d3c-468f-893b-1a962e693e71 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:19.004480 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504d3d-4d3c-468f-893b-1a962e693e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:19.031753 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-df504d3d-4d3c-468f-893b-1a962e693e71 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:19.033302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-df504d3d-4d3c-468f-893b-1a962e693e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:19.061635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:19.073726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d960f48-8374-4d8c-a933-4bc1b82d9751 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:19.155177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 251/486] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:19.255719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d960f48-8374-4d8c-a933-4bc1b82d9751 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:19.307925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a1999-c4bf-48a2-b64e-807923222d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:19.308634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a1999-c4bf-48a2-b64e-807923222d0b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:19.308915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a1999-c4bf-48a2-b64e-807923222d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:19.309188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a1999-c4bf-48a2-b64e-807923222d0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:19.322097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df504d3d-4d3c-468f-893b-1a962e693e71 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:19.332261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a1999-c4bf-48a2-b64e-807923222d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-0b4a1999-c4bf-48a2-b64e-807923222d0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:19.338086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:19.339060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:19.339950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:19.346343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 236/487] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:19.352297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a1999-c4bf-48a2-b64e-807923222d0b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:19.393440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:19.503163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df504d3d-4d3c-468f-893b-1a962e693e71 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:19.526274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a1999-c4bf-48a2-b64e-807923222d0b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:19.549311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f679d-3374-4c18-a7c8-f816d54e2f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:19.550915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f679d-3374-4c18-a7c8-f816d54e2f20 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:19.551420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f679d-3374-4c18-a7c8-f816d54e2f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:19.552137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f679d-3374-4c18-a7c8-f816d54e2f20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:19.564938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce4488-f293-46a5-9040-305b5c69777e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:19.565699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce4488-f293-46a5-9040-305b5c69777e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:19.566049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce4488-f293-46a5-9040-305b5c69777e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:19.566392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce4488-f293-46a5-9040-305b5c69777e None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:19.567154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce4488-f293-46a5-9040-305b5c69777e None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:19.567528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce4488-f293-46a5-9040-305b5c69777e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:19.568223 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f679d-3374-4c18-a7c8-f816d54e2f20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:19.569054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce4488-f293-46a5-9040-305b5c69777e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-bf9f679d-3374-4c18-a7c8-f816d54e2f20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:19.575028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:19.575938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 252/488] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:19.577157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f679d-3374-4c18-a7c8-f816d54e2f20 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:19.585614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 237/489] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:19.586576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce4488-f293-46a5-9040-305b5c69777e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:19.597751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:19.682871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:19.705147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce4488-f293-46a5-9040-305b5c69777e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:19.715485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f679d-3374-4c18-a7c8-f816d54e2f20 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:19.747085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd4cdd-8306-49ed-911c-d9a288d9bb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:19.747859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd4cdd-8306-49ed-911c-d9a288d9bb23 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:19.748314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd4cdd-8306-49ed-911c-d9a288d9bb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:19.748678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd4cdd-8306-49ed-911c-d9a288d9bb23 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:19.751690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd4cdd-8306-49ed-911c-d9a288d9bb23 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:19.751690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd4cdd-8306-49ed-911c-d9a288d9bb23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:19.751690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd4cdd-8306-49ed-911c-d9a288d9bb23 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:19.766562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 238/490] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:19.767721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd4cdd-8306-49ed-911c-d9a288d9bb23 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:19.771367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc67bf-c7e2-44ce-aa43-728baa4020de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:19.772267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc67bf-c7e2-44ce-aa43-728baa4020de None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:19.772818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc67bf-c7e2-44ce-aa43-728baa4020de None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:19.773280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc67bf-c7e2-44ce-aa43-728baa4020de None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:19.795556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc67bf-c7e2-44ce-aa43-728baa4020de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-62dc67bf-c7e2-44ce-aa43-728baa4020de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:19.801815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:19.802727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:19.803576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:19.812052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 253/491] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:19.812052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc67bf-c7e2-44ce-aa43-728baa4020de None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:19.908900 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:19.927881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd4cdd-8306-49ed-911c-d9a288d9bb23 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:19.950227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc67bf-c7e2-44ce-aa43-728baa4020de None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:19.973432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3cde6-e51e-4e0b-a753-f889791c211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:19.974074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3cde6-e51e-4e0b-a753-f889791c211a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:19.974353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3cde6-e51e-4e0b-a753-f889791c211a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:19.974653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3cde6-e51e-4e0b-a753-f889791c211a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:19.985611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:19.988296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb3cde6-e51e-4e0b-a753-f889791c211a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:19.990249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb3cde6-e51e-4e0b-a753-f889791c211a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:19.996570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbbd1ec-9a0c-4920-9ad0-1ffcc57cb5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:19.997448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbd1ec-9a0c-4920-9ad0-1ffcc57cb5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:19.997913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbd1ec-9a0c-4920-9ad0-1ffcc57cb5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:19.998392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbd1ec-9a0c-4920-9ad0-1ffcc57cb5c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-1fb3cde6-e51e-4e0b-a753-f889791c211a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:20.006844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:20.007923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:20.009064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:20.009064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:20.009064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:20.009064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 239/492] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:20.009064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3cde6-e51e-4e0b-a753-f889791c211a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:20.014479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbbd1ec-9a0c-4920-9ad0-1ffcc57cb5c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:20.016001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbbd1ec-9a0c-4920-9ad0-1ffcc57cb5c8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-dbbbd1ec-9a0c-4920-9ad0-1ffcc57cb5c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:20.032633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:20.033606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:20.035164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:20.035164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:20.035164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:20.035164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 254/493] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:20.035164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeec80-9859-46b4-b1ad-dd5dada38c04 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.035445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeec80-9859-46b4-b1ad-dd5dada38c04 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.035513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeec80-9859-46b4-b1ad-dd5dada38c04 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.036887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 255/494] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:20.037407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aeec80-9859-46b4-b1ad-dd5dada38c04 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:20.121009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3cde6-e51e-4e0b-a753-f889791c211a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:20.146036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aeec80-9859-46b4-b1ad-dd5dada38c04 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:20.157250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837b1ac-20ae-4b38-afd1-dcb5dc32993a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:20.157873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837b1ac-20ae-4b38-afd1-dcb5dc32993a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.158147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837b1ac-20ae-4b38-afd1-dcb5dc32993a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.158429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837b1ac-20ae-4b38-afd1-dcb5dc32993a None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.159139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837b1ac-20ae-4b38-afd1-dcb5dc32993a None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:20.159457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837b1ac-20ae-4b38-afd1-dcb5dc32993a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:20.160830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837b1ac-20ae-4b38-afd1-dcb5dc32993a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:20.169572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 240/495] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:20.171009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd3e4d-f678-4488-9d4a-a6dfc7839d85 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.171307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd3e4d-f678-4488-9d4a-a6dfc7839d85 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.171578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd3e4d-f678-4488-9d4a-a6dfc7839d85 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.172784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 241/496] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:20.173476 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd3e4d-f678-4488-9d4a-a6dfc7839d85 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:20.178262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:20.190050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df45153-7d40-4676-84db-4c2812044ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:20.190734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df45153-7d40-4676-84db-4c2812044ebf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.191003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df45153-7d40-4676-84db-4c2812044ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.191284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df45153-7d40-4676-84db-4c2812044ebf None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.191992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df45153-7d40-4676-84db-4c2812044ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:20.202271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df45153-7d40-4676-84db-4c2812044ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:20.202271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df45153-7d40-4676-84db-4c2812044ebf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:20.206532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 256/497] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:20.207913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55023bfd-f0da-48bb-92b2-9f2dc3507446 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.208200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55023bfd-f0da-48bb-92b2-9f2dc3507446 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.208476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55023bfd-f0da-48bb-92b2-9f2dc3507446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.227006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-55023bfd-f0da-48bb-92b2-9f2dc3507446 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:20.228190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-55023bfd-f0da-48bb-92b2-9f2dc3507446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:20.248391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:20.299512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd3e4d-f678-4488-9d4a-a6dfc7839d85 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:20.300030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 257/498] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:20.300885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55023bfd-f0da-48bb-92b2-9f2dc3507446 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:20.345162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a8d03-ef6c-420c-a435-9532769599fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:20.345947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a8d03-ef6c-420c-a435-9532769599fd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.346221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a8d03-ef6c-420c-a435-9532769599fd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.346494 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a8d03-ef6c-420c-a435-9532769599fd None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.347175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a8d03-ef6c-420c-a435-9532769599fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:20.347472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a8d03-ef6c-420c-a435-9532769599fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:20.349071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a8d03-ef6c-420c-a435-9532769599fd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:20.362661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 242/499] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:20.364053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a8cfb-6614-4bf3-b10e-3a33310223d8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.364338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a8cfb-6614-4bf3-b10e-3a33310223d8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.364675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a8cfb-6614-4bf3-b10e-3a33310223d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.386433 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-678a8cfb-6614-4bf3-b10e-3a33310223d8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:20.387688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-678a8cfb-6614-4bf3-b10e-3a33310223d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:20.412032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55023bfd-f0da-48bb-92b2-9f2dc3507446 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:20.430412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:20.456218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a74f0-e078-400f-9eae-e37374bace2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:20.456862 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a74f0-e078-400f-9eae-e37374bace2d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.457153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a74f0-e078-400f-9eae-e37374bace2d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.457411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a74f0-e078-400f-9eae-e37374bace2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.474986 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 243/500] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:20.474986 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a8cfb-6614-4bf3-b10e-3a33310223d8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:20.475286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a74f0-e078-400f-9eae-e37374bace2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:20.475286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a74f0-e078-400f-9eae-e37374bace2d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-7d8a74f0-e078-400f-9eae-e37374bace2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:20.493405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:20.494369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:20.495255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:20.495255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:20.495255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:20.495255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 258/501] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:20.495494 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a74f0-e078-400f-9eae-e37374bace2d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:20.532323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:20.611201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a8cfb-6614-4bf3-b10e-3a33310223d8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:20.627945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a74f0-e078-400f-9eae-e37374bace2d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:20.646324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15aa2d-f3d0-47f1-8e55-9bf59047d8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:20.646981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15aa2d-f3d0-47f1-8e55-9bf59047d8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.647265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15aa2d-f3d0-47f1-8e55-9bf59047d8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.647536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15aa2d-f3d0-47f1-8e55-9bf59047d8c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.655607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15aa2d-f3d0-47f1-8e55-9bf59047d8c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:20.657055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15aa2d-f3d0-47f1-8e55-9bf59047d8c3 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:20.666846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c320fd-9007-4a11-920a-489390b9f89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:20.667507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c320fd-9007-4a11-920a-489390b9f89b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.668173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c320fd-9007-4a11-920a-489390b9f89b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.668450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c320fd-9007-4a11-920a-489390b9f89b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-aa15aa2d-f3d0-47f1-8e55-9bf59047d8c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:20.670937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:20.671843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:20.672723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:20.672723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:20.672723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:20.672723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:20.672723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:20.672723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:20.672723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:20.672723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:20.672723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:20.672723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:20.672723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 244/502] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 22 16:53:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:20.673372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15aa2d-f3d0-47f1-8e55-9bf59047d8c3 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:20.682444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c320fd-9007-4a11-920a-489390b9f89b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:20.684119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c320fd-9007-4a11-920a-489390b9f89b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-17c320fd-9007-4a11-920a-489390b9f89b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:20.699935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:20.700893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:20.701772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:20.701772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:20.701772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:20.701772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:20.701772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:20.701772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:20.701772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:20.701772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:20.701772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 16:53:20.701772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:20.704685 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 259/503] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:20.704685 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c320fd-9007-4a11-920a-489390b9f89b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:20.736301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:20.788760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15aa2d-f3d0-47f1-8e55-9bf59047d8c3 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:20.791118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:20.817263 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c320fd-9007-4a11-920a-489390b9f89b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:20.825631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac62dc-3a90-49c9-b9b6-9deaed080ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:20.826237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac62dc-3a90-49c9-b9b6-9deaed080ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.826515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac62dc-3a90-49c9-b9b6-9deaed080ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.826820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac62dc-3a90-49c9-b9b6-9deaed080ac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.827461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac62dc-3a90-49c9-b9b6-9deaed080ac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:20.827761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac62dc-3a90-49c9-b9b6-9deaed080ac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:20.829134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac62dc-3a90-49c9-b9b6-9deaed080ac3 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:20.842308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 245/504] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:20.843172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac62dc-3a90-49c9-b9b6-9deaed080ac3 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:20.852954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e433dad-0c1a-4298-90c6-447409d38c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:20.853587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433dad-0c1a-4298-90c6-447409d38c4a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.853864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433dad-0c1a-4298-90c6-447409d38c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.854134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433dad-0c1a-4298-90c6-447409d38c4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.864560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e433dad-0c1a-4298-90c6-447409d38c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-3e433dad-0c1a-4298-90c6-447409d38c4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:20.869819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:20.870896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 260/505] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:20.871883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e433dad-0c1a-4298-90c6-447409d38c4a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:20.946268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac62dc-3a90-49c9-b9b6-9deaed080ac3 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:20.957945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:20.974510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e433dad-0c1a-4298-90c6-447409d38c4a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:20.982410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e70d7-187d-45a8-9de0-20ec3667c432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:20.983045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e70d7-187d-45a8-9de0-20ec3667c432 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:20.983329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e70d7-187d-45a8-9de0-20ec3667c432 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:20.983599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e70d7-187d-45a8-9de0-20ec3667c432 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:20.984323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e70d7-187d-45a8-9de0-20ec3667c432 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:20.984629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e70d7-187d-45a8-9de0-20ec3667c432 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:20.986073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e70d7-187d-45a8-9de0-20ec3667c432 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:21.009562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316690cc-acae-4a47-bb24-43c265c9b47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:21.010157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316690cc-acae-4a47-bb24-43c265c9b47d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:21.010435 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316690cc-acae-4a47-bb24-43c265c9b47d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:21.010703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316690cc-acae-4a47-bb24-43c265c9b47d None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:21.012101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316690cc-acae-4a47-bb24-43c265c9b47d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:21.012411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316690cc-acae-4a47-bb24-43c265c9b47d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:21.013855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316690cc-acae-4a47-bb24-43c265c9b47d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:21.024573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 246/506] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 22 16:53:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:21.025433 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e70d7-187d-45a8-9de0-20ec3667c432 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:21.050446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:21.051669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 261/507] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 22 16:53:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:21.052431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316690cc-acae-4a47-bb24-43c265c9b47d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:21.137311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e70d7-187d-45a8-9de0-20ec3667c432 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:21.165008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316690cc-acae-4a47-bb24-43c265c9b47d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:21.172829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cfd79-def3-46a4-bbcc-9134a24a12ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:21.174150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cfd79-def3-46a4-bbcc-9134a24a12ed None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:21.174431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cfd79-def3-46a4-bbcc-9134a24a12ed None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:21.174705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cfd79-def3-46a4-bbcc-9134a24a12ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:21.185507 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cfd79-def3-46a4-bbcc-9134a24a12ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:21.187008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cfd79-def3-46a4-bbcc-9134a24a12ed None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:21.202642 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9aba6e-f2cb-49c2-bef0-7d6a7b5ed582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:21.203308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aba6e-f2cb-49c2-bef0-7d6a7b5ed582 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f49cfd79-def3-46a4-bbcc-9134a24a12ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:21.203422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:21.204531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:21.205912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:21.205912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:21.205912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:21.206100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aba6e-f2cb-49c2-bef0-7d6a7b5ed582 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:21.206100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aba6e-f2cb-49c2-bef0-7d6a7b5ed582 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:21.206224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 247/508] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:21.206224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cfd79-def3-46a4-bbcc-9134a24a12ed None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:21.214199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9aba6e-f2cb-49c2-bef0-7d6a7b5ed582 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:21.216285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9aba6e-f2cb-49c2-bef0-7d6a7b5ed582 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-9b9aba6e-f2cb-49c2-bef0-7d6a7b5ed582 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:21.228544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:21.229617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:21.230532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:21.230532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:21.230532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:21.230532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:21.230532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:21.230532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:21.230532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:21.230532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:21.230532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 16:53:21.230532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:21.230532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 262/509] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:21.231843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9aba6e-f2cb-49c2-bef0-7d6a7b5ed582 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:21.235653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:21.286418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:21.311968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cfd79-def3-46a4-bbcc-9134a24a12ed None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:21.335410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9aba6e-f2cb-49c2-bef0-7d6a7b5ed582 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:21.346721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da79cfc-1d1d-4dfd-aa26-4d1734603e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:21.347360 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79cfc-1d1d-4dfd-aa26-4d1734603e30 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:21.347642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79cfc-1d1d-4dfd-aa26-4d1734603e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:21.347942 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79cfc-1d1d-4dfd-aa26-4d1734603e30 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:21.348605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da79cfc-1d1d-4dfd-aa26-4d1734603e30 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:21.348907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da79cfc-1d1d-4dfd-aa26-4d1734603e30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:21.350297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da79cfc-1d1d-4dfd-aa26-4d1734603e30 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:21.364516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 248/510] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:21.364516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da79cfc-1d1d-4dfd-aa26-4d1734603e30 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:21.377003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2ddc5-8183-42f3-a1cd-68cfe24f676c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:21.377612 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2ddc5-8183-42f3-a1cd-68cfe24f676c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:21.377880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2ddc5-8183-42f3-a1cd-68cfe24f676c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:21.378155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2ddc5-8183-42f3-a1cd-68cfe24f676c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:21.447945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2ddc5-8183-42f3-a1cd-68cfe24f676c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:21.449459 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2ddc5-8183-42f3-a1cd-68cfe24f676c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:21.452203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:21.466251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da79cfc-1d1d-4dfd-aa26-4d1734603e30 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:21.502257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8679379-5ebd-4391-a2df-074cc38fea9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:21.502871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8679379-5ebd-4391-a2df-074cc38fea9b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:21.503176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8679379-5ebd-4391-a2df-074cc38fea9b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:21.503457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8679379-5ebd-4391-a2df-074cc38fea9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:21.507619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:21.556243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 263/511] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:21.556925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2ddc5-8183-42f3-a1cd-68cfe24f676c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:21.577691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8679379-5ebd-4391-a2df-074cc38fea9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:21.579195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8679379-5ebd-4391-a2df-074cc38fea9b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:21.664483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2ddc5-8183-42f3-a1cd-68cfe24f676c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:21.666764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:21.689462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 249/512] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:21.689462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8679379-5ebd-4391-a2df-074cc38fea9b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:21.702017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbbec7-4f52-4e65-9390-79807a263522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:21.702545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbbec7-4f52-4e65-9390-79807a263522 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:21.702823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbbec7-4f52-4e65-9390-79807a263522 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:21.703123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbbec7-4f52-4e65-9390-79807a263522 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:21.703759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbbec7-4f52-4e65-9390-79807a263522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:21.704056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbbec7-4f52-4e65-9390-79807a263522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:21.705419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbbec7-4f52-4e65-9390-79807a263522 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:21.717044 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 264/513] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:21.717858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbbec7-4f52-4e65-9390-79807a263522 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:21.726861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:21.801891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8679379-5ebd-4391-a2df-074cc38fea9b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:21.820446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbbec7-4f52-4e65-9390-79807a263522 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:21.836412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f55412-9646-4da8-bb05-6b71e11a016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:21.837045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f55412-9646-4da8-bb05-6b71e11a016a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:21.837322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f55412-9646-4da8-bb05-6b71e11a016a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:21.837594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f55412-9646-4da8-bb05-6b71e11a016a None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:21.838323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f55412-9646-4da8-bb05-6b71e11a016a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:21.838624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f55412-9646-4da8-bb05-6b71e11a016a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:21.841230 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f55412-9646-4da8-bb05-6b71e11a016a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:21.854432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3632fee-9829-4cdb-9695-f7253a124e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:21.855102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3632fee-9829-4cdb-9695-f7253a124e91 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:21.855902 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3632fee-9829-4cdb-9695-f7253a124e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:21.856169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3632fee-9829-4cdb-9695-f7253a124e91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:21.859318 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 250/514] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 22 16:53:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:21.864198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f55412-9646-4da8-bb05-6b71e11a016a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:21.868096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3632fee-9829-4cdb-9695-f7253a124e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:21.869582 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3632fee-9829-4cdb-9695-f7253a124e91 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e3632fee-9829-4cdb-9695-f7253a124e91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:21.883496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:21.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:21.885374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:21.885374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:21.885374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:21.885374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 265/515] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:21.885374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3632fee-9829-4cdb-9695-f7253a124e91 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:21.904696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:21.964301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:21.971854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f55412-9646-4da8-bb05-6b71e11a016a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:21.986348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3632fee-9829-4cdb-9695-f7253a124e91 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:22.005749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e759-b8dc-4913-9368-1b4d077ff7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:22.006447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e759-b8dc-4913-9368-1b4d077ff7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:22.006727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e759-b8dc-4913-9368-1b4d077ff7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:22.007009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e759-b8dc-4913-9368-1b4d077ff7c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:22.017301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102e759-b8dc-4913-9368-1b4d077ff7c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:22.018729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102e759-b8dc-4913-9368-1b4d077ff7c2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:22.018970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00882e12-86a9-4270-8291-66dc65532487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:22.019596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882e12-86a9-4270-8291-66dc65532487 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:22.019882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882e12-86a9-4270-8291-66dc65532487 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:22.020154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882e12-86a9-4270-8291-66dc65532487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:22.031254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00882e12-86a9-4270-8291-66dc65532487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:22.032677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00882e12-86a9-4270-8291-66dc65532487 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-a102e759-b8dc-4913-9368-1b4d077ff7c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:22.034190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:22.035284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:22.036574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:22.036574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:22.036574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:22.036574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 251/516] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:22.036574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e759-b8dc-4913-9368-1b4d077ff7c2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-00882e12-86a9-4270-8291-66dc65532487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:22.050981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:22.051933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:22.052784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:22.052784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:22.052784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:22.052784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 266/517] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:22.052993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00882e12-86a9-4270-8291-66dc65532487 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:22.126539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:22.146730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e759-b8dc-4913-9368-1b4d077ff7c2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:22.163338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00882e12-86a9-4270-8291-66dc65532487 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:22.181885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796049e4-f688-4b4b-b8f5-1989cedf29f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:22.182512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796049e4-f688-4b4b-b8f5-1989cedf29f2 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:22.182791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796049e4-f688-4b4b-b8f5-1989cedf29f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:22.183098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796049e4-f688-4b4b-b8f5-1989cedf29f2 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:22.183768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796049e4-f688-4b4b-b8f5-1989cedf29f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:22.185257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796049e4-f688-4b4b-b8f5-1989cedf29f2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:22.200875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8301e9-907b-4717-b0e8-0675e5e41129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:22.201504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8301e9-907b-4717-b0e8-0675e5e41129 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:22.201786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8301e9-907b-4717-b0e8-0675e5e41129 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:22.202059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8301e9-907b-4717-b0e8-0675e5e41129 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:22.209516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 252/518] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:53:22 2022] POST /identity/v3/users => generated 247 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:53:22.210422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796049e4-f688-4b4b-b8f5-1989cedf29f2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:22.214275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:22.293540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8301e9-907b-4717-b0e8-0675e5e41129 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:22.295367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8301e9-907b-4717-b0e8-0675e5e41129 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:22.341416 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796049e4-f688-4b4b-b8f5-1989cedf29f2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:22.383704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82cefa-b9d0-4de2-88ef-9506d7edee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:22.383982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:22.384331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82cefa-b9d0-4de2-88ef-9506d7edee31 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:22.384611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82cefa-b9d0-4de2-88ef-9506d7edee31 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:22.384921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82cefa-b9d0-4de2-88ef-9506d7edee31 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:22.385644 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82cefa-b9d0-4de2-88ef-9506d7edee31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:22.386243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82cefa-b9d0-4de2-88ef-9506d7edee31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:22.388675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82cefa-b9d0-4de2-88ef-9506d7edee31 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:22.402454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 253/519] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:22.403077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82cefa-b9d0-4de2-88ef-9506d7edee31 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:22.433149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 267/520] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:22.434217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8301e9-907b-4717-b0e8-0675e5e41129 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:22.481301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:22.526387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82cefa-b9d0-4de2-88ef-9506d7edee31 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:22.546689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8301e9-907b-4717-b0e8-0675e5e41129 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:22.564791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d437d4-c80d-4268-a3d4-a547ae8a6055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:22.565641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d437d4-c80d-4268-a3d4-a547ae8a6055 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:22.566102 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d437d4-c80d-4268-a3d4-a547ae8a6055 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:22.566374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d437d4-c80d-4268-a3d4-a547ae8a6055 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:22.567050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d437d4-c80d-4268-a3d4-a547ae8a6055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:22.567355 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d437d4-c80d-4268-a3d4-a547ae8a6055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:22.568775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d437d4-c80d-4268-a3d4-a547ae8a6055 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:22.581404 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 254/521] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:22.582076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d437d4-c80d-4268-a3d4-a547ae8a6055 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:22.585758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ddcea-a43b-40fc-b031-5798cffff3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:22.587603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ddcea-a43b-40fc-b031-5798cffff3de None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:22.588096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ddcea-a43b-40fc-b031-5798cffff3de None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:22.588566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ddcea-a43b-40fc-b031-5798cffff3de None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:22.589402 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ddcea-a43b-40fc-b031-5798cffff3de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:22.590464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ddcea-a43b-40fc-b031-5798cffff3de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:22.592745 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ddcea-a43b-40fc-b031-5798cffff3de None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:22.609435 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 268/522] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:22.610376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ddcea-a43b-40fc-b031-5798cffff3de None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:22.647937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:22.702051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d437d4-c80d-4268-a3d4-a547ae8a6055 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:22.726677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:22.735774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ddcea-a43b-40fc-b031-5798cffff3de None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:22.741054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae08edd-ae4a-415b-9a75-f1474b98ca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:22.741691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae08edd-ae4a-415b-9a75-f1474b98ca5a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:22.741970 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae08edd-ae4a-415b-9a75-f1474b98ca5a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:22.742241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae08edd-ae4a-415b-9a75-f1474b98ca5a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:22.753408 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae08edd-ae4a-415b-9a75-f1474b98ca5a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:22.755178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae08edd-ae4a-415b-9a75-f1474b98ca5a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-0ae08edd-ae4a-415b-9a75-f1474b98ca5a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:22.774174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:22.775452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 255/523] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:22.781938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae08edd-ae4a-415b-9a75-f1474b98ca5a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:22.783666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0c0e7-00db-417a-bb74-04e5b403428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:22.784309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0c0e7-00db-417a-bb74-04e5b403428b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:22.784585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0c0e7-00db-417a-bb74-04e5b403428b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:22.784851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0c0e7-00db-417a-bb74-04e5b403428b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:22.798906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0c0e7-00db-417a-bb74-04e5b403428b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:22.800337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0c0e7-00db-417a-bb74-04e5b403428b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-f9f0c0e7-00db-417a-bb74-04e5b403428b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:22.817735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:22.823383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:22.824516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:22.824516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:22.824516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:22.824516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 269/524] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:22.824516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0c0e7-00db-417a-bb74-04e5b403428b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:22.888068 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae08edd-ae4a-415b-9a75-f1474b98ca5a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:22.912351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:22.934816 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9945bc5-34db-4a1c-a7b9-4b76f0f88406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:22.935502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9945bc5-34db-4a1c-a7b9-4b76f0f88406 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:22.935776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9945bc5-34db-4a1c-a7b9-4b76f0f88406 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:22.936038 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9945bc5-34db-4a1c-a7b9-4b76f0f88406 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:22.936816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0c0e7-00db-417a-bb74-04e5b403428b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:22.943211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9945bc5-34db-4a1c-a7b9-4b76f0f88406 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:22.944728 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9945bc5-34db-4a1c-a7b9-4b76f0f88406 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-b9945bc5-34db-4a1c-a7b9-4b76f0f88406 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:22.957152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:22.958238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:22.963938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:22.963938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:22.963938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:22.963938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:22.963938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:22.963938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:22.963938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:22.963938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:22.963938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 256/525] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:22.963938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9945bc5-34db-4a1c-a7b9-4b76f0f88406 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:22.981249 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c41ce-eec2-44b8-993a-df204d401931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:22.981892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c41ce-eec2-44b8-993a-df204d401931 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:22.982173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c41ce-eec2-44b8-993a-df204d401931 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:22.982446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c41ce-eec2-44b8-993a-df204d401931 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:22.990336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c41ce-eec2-44b8-993a-df204d401931 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:22.991954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c41ce-eec2-44b8-993a-df204d401931 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e91c41ce-eec2-44b8-993a-df204d401931 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:23.006343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:23.007670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:23.008525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:23.008525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:23.008525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:23.008525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:23.008525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:23.008525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:23.008525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:23.008525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:23.008525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 270/526] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:23.008525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c41ce-eec2-44b8-993a-df204d401931 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:23.011851 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:23.074350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9945bc5-34db-4a1c-a7b9-4b76f0f88406 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:23.110719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb786c-748a-4819-960d-6a2cfde740b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:23.111518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb786c-748a-4819-960d-6a2cfde740b7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:23.111807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb786c-748a-4819-960d-6a2cfde740b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:23.112084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb786c-748a-4819-960d-6a2cfde740b7 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:23.112834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb786c-748a-4819-960d-6a2cfde740b7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:23.113142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb786c-748a-4819-960d-6a2cfde740b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:23.114664 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb786c-748a-4819-960d-6a2cfde740b7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:23.125882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c41ce-eec2-44b8-993a-df204d401931 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:23.140841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 257/527] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Sep 22 16:53:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:23.140841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb786c-748a-4819-960d-6a2cfde740b7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:23.169059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c73e6-6fb8-4368-8481-e7eb3860bdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:23.169677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c73e6-6fb8-4368-8481-e7eb3860bdb6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:23.169954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c73e6-6fb8-4368-8481-e7eb3860bdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:23.170222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c73e6-6fb8-4368-8481-e7eb3860bdb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:23.170878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c73e6-6fb8-4368-8481-e7eb3860bdb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:23.171208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c73e6-6fb8-4368-8481-e7eb3860bdb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:23.173197 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c73e6-6fb8-4368-8481-e7eb3860bdb6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:23.185418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:23.189958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 271/528] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:23.190815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c73e6-6fb8-4368-8481-e7eb3860bdb6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:23.256308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:23.259540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb786c-748a-4819-960d-6a2cfde740b7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:23.297285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a2131-0a05-4e4b-b593-69c5dd4a9c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:23.297908 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a2131-0a05-4e4b-b593-69c5dd4a9c6b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:23.298187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a2131-0a05-4e4b-b593-69c5dd4a9c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:23.298440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a2131-0a05-4e4b-b593-69c5dd4a9c6b None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:23.299421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a2131-0a05-4e4b-b593-69c5dd4a9c6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:23.299718 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a2131-0a05-4e4b-b593-69c5dd4a9c6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:23.301138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a2131-0a05-4e4b-b593-69c5dd4a9c6b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:23.320014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 258/529] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:23.320876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a2131-0a05-4e4b-b593-69c5dd4a9c6b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:23.357829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c73e6-6fb8-4368-8481-e7eb3860bdb6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:23.416010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785128af-c958-4fe4-b131-fa7dcbb757df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:23.416545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785128af-c958-4fe4-b131-fa7dcbb757df None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:23.416824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785128af-c958-4fe4-b131-fa7dcbb757df None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:23.417090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785128af-c958-4fe4-b131-fa7dcbb757df None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:23.418244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785128af-c958-4fe4-b131-fa7dcbb757df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:23.418244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785128af-c958-4fe4-b131-fa7dcbb757df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:23.424499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785128af-c958-4fe4-b131-fa7dcbb757df None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:23.437148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a2131-0a05-4e4b-b593-69c5dd4a9c6b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:23.447334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 272/530] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:23.448065 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785128af-c958-4fe4-b131-fa7dcbb757df None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:23.474631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702be5c3-badb-442a-9731-6da206724c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:23.475263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702be5c3-badb-442a-9731-6da206724c07 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:23.475643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702be5c3-badb-442a-9731-6da206724c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:23.475910 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702be5c3-badb-442a-9731-6da206724c07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:23.507228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:23.509407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:23.553837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785128af-c958-4fe4-b131-fa7dcbb757df None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:23.554101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702be5c3-badb-442a-9731-6da206724c07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:23.555570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702be5c3-badb-442a-9731-6da206724c07 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:23.596638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735e99-97db-4c2d-9f70-8975ffde862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:23.597704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735e99-97db-4c2d-9f70-8975ffde862e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:23.597984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735e99-97db-4c2d-9f70-8975ffde862e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:23.598253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735e99-97db-4c2d-9f70-8975ffde862e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:23.606455 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46735e99-97db-4c2d-9f70-8975ffde862e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:23.608383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46735e99-97db-4c2d-9f70-8975ffde862e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-46735e99-97db-4c2d-9f70-8975ffde862e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:23.620068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:23.621346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:23.622698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:23.622698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:23.622698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:23.622698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:23.622698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:23.622698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:23.622698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:23.622698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:23.627102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 273/531] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:23.628066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735e99-97db-4c2d-9f70-8975ffde862e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:23.673962 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 259/532] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:23.675022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702be5c3-badb-442a-9731-6da206724c07 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:23.737862 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:23.748095 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735e99-97db-4c2d-9f70-8975ffde862e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:23.782973 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:23.798887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702be5c3-badb-442a-9731-6da206724c07 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:23.802560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93d355-947d-493b-b6cd-f6a6ecf40da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:23.803199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93d355-947d-493b-b6cd-f6a6ecf40da2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:23.803609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93d355-947d-493b-b6cd-f6a6ecf40da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:23.804025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93d355-947d-493b-b6cd-f6a6ecf40da2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:23.805149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93d355-947d-493b-b6cd-f6a6ecf40da2 None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=f6c1ce47d8704111988a78e625fd968c) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:23.805680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93d355-947d-493b-b6cd-f6a6ecf40da2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:23.808543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93d355-947d-493b-b6cd-f6a6ecf40da2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:23.826977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 274/533] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:23 2022] GET /identity/v3/role_assignments?user.id=f6c1ce47d8704111988a78e625fd968c&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:23.826977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93d355-947d-493b-b6cd-f6a6ecf40da2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:23.855953 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee390c-0ee5-4415-a86a-d893566d0d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:23.858025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee390c-0ee5-4415-a86a-d893566d0d6a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:23.858439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee390c-0ee5-4415-a86a-d893566d0d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:23.858725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee390c-0ee5-4415-a86a-d893566d0d6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:23.859645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee390c-0ee5-4415-a86a-d893566d0d6a None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=ae1b8a51c9fa4f2594bdd79b58962b0a) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:23.860339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee390c-0ee5-4415-a86a-d893566d0d6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:23.863089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee390c-0ee5-4415-a86a-d893566d0d6a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:23.872103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 260/534] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:23 2022] GET /identity/v3/role_assignments?user.id=ae1b8a51c9fa4f2594bdd79b58962b0a&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:23.873126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee390c-0ee5-4415-a86a-d893566d0d6a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:23.982081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93d355-947d-493b-b6cd-f6a6ecf40da2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:24.014914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee390c-0ee5-4415-a86a-d893566d0d6a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:24.062445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:24.072742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ce79-8eae-4430-b46c-9fda282b3d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:24.073625 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ce79-8eae-4430-b46c-9fda282b3d94 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:24.074197 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ce79-8eae-4430-b46c-9fda282b3d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:24.074724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ce79-8eae-4430-b46c-9fda282b3d94 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:24.075636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed98781-a9fb-4af7-ad3c-2586e6ee306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:24.075812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316ce79-8eae-4430-b46c-9fda282b3d94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:24.076257 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316ce79-8eae-4430-b46c-9fda282b3d94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:24.076383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed98781-a9fb-4af7-ad3c-2586e6ee306c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:24.076671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed98781-a9fb-4af7-ad3c-2586e6ee306c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:24.076946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed98781-a9fb-4af7-ad3c-2586e6ee306c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:24.078311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316ce79-8eae-4430-b46c-9fda282b3d94 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:24.097399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed98781-a9fb-4af7-ad3c-2586e6ee306c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:24.099310 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed98781-a9fb-4af7-ad3c-2586e6ee306c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:24.101001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 275/535] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:24.108378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ce79-8eae-4430-b46c-9fda282b3d94 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-aed98781-a9fb-4af7-ad3c-2586e6ee306c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:24.117702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:24.119083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:24.119938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:24.119938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:24.119938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:24.123073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 261/536] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:24.126732 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed98781-a9fb-4af7-ad3c-2586e6ee306c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:24.221236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:24.289633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed98781-a9fb-4af7-ad3c-2586e6ee306c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:24.329420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05087910-dcf1-46bd-abad-1aec0fb6862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:24.330383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05087910-dcf1-46bd-abad-1aec0fb6862c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:24.330967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05087910-dcf1-46bd-abad-1aec0fb6862c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:24.331457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05087910-dcf1-46bd-abad-1aec0fb6862c None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:24.332603 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05087910-dcf1-46bd-abad-1aec0fb6862c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:24.333106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05087910-dcf1-46bd-abad-1aec0fb6862c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:24.334701 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05087910-dcf1-46bd-abad-1aec0fb6862c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:24.353382 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ce79-8eae-4430-b46c-9fda282b3d94 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:24.353880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 262/537] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:24.355516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05087910-dcf1-46bd-abad-1aec0fb6862c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:24.407718 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:24.424819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53190d-b3b3-493b-a794-6065386337e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:24.425842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53190d-b3b3-493b-a794-6065386337e4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:24.426322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53190d-b3b3-493b-a794-6065386337e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:24.426804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53190d-b3b3-493b-a794-6065386337e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:24.427759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53190d-b3b3-493b-a794-6065386337e4 None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=6103e9a612c84a9ea7d21cc5128fa40b) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:24.430085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53190d-b3b3-493b-a794-6065386337e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:24.431846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53190d-b3b3-493b-a794-6065386337e4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:24.447852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 276/538] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:24 2022] GET /identity/v3/role_assignments?user.id=6103e9a612c84a9ea7d21cc5128fa40b&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 259 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:24.502030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05087910-dcf1-46bd-abad-1aec0fb6862c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:24.575398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef39147-9225-494d-812f-83f07dca561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:24.576088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef39147-9225-494d-812f-83f07dca561d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:24.576571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef39147-9225-494d-812f-83f07dca561d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:24.576859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef39147-9225-494d-812f-83f07dca561d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:24.590873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef39147-9225-494d-812f-83f07dca561d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:24.592514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef39147-9225-494d-812f-83f07dca561d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-eef39147-9225-494d-812f-83f07dca561d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:24.605604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:24.606518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:24.607783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:24.607783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:24.607783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:24.607783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:24.607783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:24.607783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:24.607783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:24.607783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:24.607783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 263/539] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:24.612843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53190d-b3b3-493b-a794-6065386337e4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:24.715972 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad34edb-7682-4322-b949-c9f74d80503f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:24.716298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad34edb-7682-4322-b949-c9f74d80503f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:24.716621 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad34edb-7682-4322-b949-c9f74d80503f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:24.718144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 264/540] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:53:24.724017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821bae3-7cbc-4ddd-be4d-4c93394beada None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:24.724331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821bae3-7cbc-4ddd-be4d-4c93394beada None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:24.724562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821bae3-7cbc-4ddd-be4d-4c93394beada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:24.759581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8821bae3-7cbc-4ddd-be4d-4c93394beada None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:24.760789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8821bae3-7cbc-4ddd-be4d-4c93394beada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:24.775693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53190d-b3b3-493b-a794-6065386337e4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:24.785812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:24.814225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a6a51-99c0-43f3-b52e-fca9191d9785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:24.814930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a6a51-99c0-43f3-b52e-fca9191d9785 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:24.815304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a6a51-99c0-43f3-b52e-fca9191d9785 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:24.815647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a6a51-99c0-43f3-b52e-fca9191d9785 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:24.816409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a6a51-99c0-43f3-b52e-fca9191d9785 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:24.816784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a6a51-99c0-43f3-b52e-fca9191d9785 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:24.818314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a6a51-99c0-43f3-b52e-fca9191d9785 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:24.832548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 277/541] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:24.839530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a6a51-99c0-43f3-b52e-fca9191d9785 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:24.918978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 265/542] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:25.037576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a6a51-99c0-43f3-b52e-fca9191d9785 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:25.115668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc68ca-d7be-431a-9004-66fb3afb5150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:25.116285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc68ca-d7be-431a-9004-66fb3afb5150 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:25.116565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc68ca-d7be-431a-9004-66fb3afb5150 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:25.116835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc68ca-d7be-431a-9004-66fb3afb5150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:25.117561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc68ca-d7be-431a-9004-66fb3afb5150 None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=8c2ed1eb664348e384824f39ca90c950) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:25.117866 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc68ca-d7be-431a-9004-66fb3afb5150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:25.119315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc68ca-d7be-431a-9004-66fb3afb5150 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:25.126118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 278/543] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:24 2022] GET /identity/v3/role_assignments?user.id=8c2ed1eb664348e384824f39ca90c950&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:25.264386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821bae3-7cbc-4ddd-be4d-4c93394beada None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:25.309105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:25.405771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821bae3-7cbc-4ddd-be4d-4c93394beada None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:25.447257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f89f69-dcd4-4f08-9774-47d236882da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:25.447902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89f69-dcd4-4f08-9774-47d236882da1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:25.448183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89f69-dcd4-4f08-9774-47d236882da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:25.448526 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89f69-dcd4-4f08-9774-47d236882da1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:25.475848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f89f69-dcd4-4f08-9774-47d236882da1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-25f89f69-dcd4-4f08-9774-47d236882da1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:25.481582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:25.482492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:25.483758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 266/544] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:25.490536 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc68ca-d7be-431a-9004-66fb3afb5150 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:25.556026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:25.682423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc68ca-d7be-431a-9004-66fb3afb5150 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:25.720860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae69925-a8b5-4364-adeb-9bcab064c72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:25.721484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae69925-a8b5-4364-adeb-9bcab064c72e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:25.721765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae69925-a8b5-4364-adeb-9bcab064c72e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:25.722035 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae69925-a8b5-4364-adeb-9bcab064c72e None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:25.722719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae69925-a8b5-4364-adeb-9bcab064c72e None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:25.723058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae69925-a8b5-4364-adeb-9bcab064c72e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:25.724490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae69925-a8b5-4364-adeb-9bcab064c72e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:25.738298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 279/545] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:25.744411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f89f69-dcd4-4f08-9774-47d236882da1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:25.863636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:25.885031 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f89f69-dcd4-4f08-9774-47d236882da1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:25.931644 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c5444-1725-4d42-945d-dbcd8ca4380e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:25.932291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c5444-1725-4d42-945d-dbcd8ca4380e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:25.932632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c5444-1725-4d42-945d-dbcd8ca4380e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:25.932849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c5444-1725-4d42-945d-dbcd8ca4380e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:25.948432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c5444-1725-4d42-945d-dbcd8ca4380e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:25.950458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c5444-1725-4d42-945d-dbcd8ca4380e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-497c5444-1725-4d42-945d-dbcd8ca4380e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:25.972786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:25.973756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:25.974867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:25.974867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:25.974867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:25.974867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 267/546] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:25.979783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae69925-a8b5-4364-adeb-9bcab064c72e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:26.124874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:26.131965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae69925-a8b5-4364-adeb-9bcab064c72e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:26.184843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424a56a-9d21-4f3e-a42f-0a94a875f4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:26.185795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424a56a-9d21-4f3e-a42f-0a94a875f4ff None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:26.186280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424a56a-9d21-4f3e-a42f-0a94a875f4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:26.186739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424a56a-9d21-4f3e-a42f-0a94a875f4ff None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:26.187596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424a56a-9d21-4f3e-a42f-0a94a875f4ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:26.188093 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424a56a-9d21-4f3e-a42f-0a94a875f4ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:26.189684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424a56a-9d21-4f3e-a42f-0a94a875f4ff None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:26.208384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 280/547] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:26.214042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c5444-1725-4d42-945d-dbcd8ca4380e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:26.367585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c5444-1725-4d42-945d-dbcd8ca4380e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:26.414188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f0e78-4e85-40f1-92e2-bbed8cead8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:26.414834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f0e78-4e85-40f1-92e2-bbed8cead8ce None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:26.415139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f0e78-4e85-40f1-92e2-bbed8cead8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:26.415411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f0e78-4e85-40f1-92e2-bbed8cead8ce None admin] PATH_INFO: `/v3/users/swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:26.428056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f0e78-4e85-40f1-92e2-bbed8cead8ce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:26.429525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f0e78-4e85-40f1-92e2-bbed8cead8ce None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:26.431485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-b48f0e78-4e85-40f1-92e2-bbed8cead8ce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:26.443350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:26.444724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:26.445776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:26.445776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:26.445776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:26.445776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:26.445776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:26.445776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:26.445776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:26.445776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:26.445776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:26.445776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:26.445776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 268/548] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:26.450761 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424a56a-9d21-4f3e-a42f-0a94a875f4ff None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:26.609749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424a56a-9d21-4f3e-a42f-0a94a875f4ff None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:26.618928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30236498-8309-4297-8d98-b1c68972ea1f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:26.618928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30236498-8309-4297-8d98-b1c68972ea1f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:26.618928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30236498-8309-4297-8d98-b1c68972ea1f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:26.619673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 269/549] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:26.624998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6c29-7f9f-4a2f-bf23-140c75cae309 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:26.625586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6c29-7f9f-4a2f-bf23-140c75cae309 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:26.626056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6c29-7f9f-4a2f-bf23-140c75cae309 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:26.627936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 270/550] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:53:26.633554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f260-ece8-4044-bd0a-a670e4b3d025 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:26.634158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f260-ece8-4044-bd0a-a670e4b3d025 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:26.634626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f260-ece8-4044-bd0a-a670e4b3d025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:26.654038 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-51b7f260-ece8-4044-bd0a-a670e4b3d025 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:26.655770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-51b7f260-ece8-4044-bd0a-a670e4b3d025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:26.656597 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c50d-de38-4a13-8cb3-f59e4b910332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:26.657214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c50d-de38-4a13-8cb3-f59e4b910332 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:26.657493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c50d-de38-4a13-8cb3-f59e4b910332 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:26.657770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c50d-de38-4a13-8cb3-f59e4b910332 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:26.658513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c50d-de38-4a13-8cb3-f59e4b910332 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:26.658821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c50d-de38-4a13-8cb3-f59e4b910332 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:26.661023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c50d-de38-4a13-8cb3-f59e4b910332 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:26.686549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 281/551] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 22 16:53:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:26.688744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a47939-f43d-4a23-a17e-fa840871c946 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:26.689054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a47939-f43d-4a23-a17e-fa840871c946 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:26.689334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a47939-f43d-4a23-a17e-fa840871c946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:26.709832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d2a47939-f43d-4a23-a17e-fa840871c946 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:26.710983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d2a47939-f43d-4a23-a17e-fa840871c946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:26.720424 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:26.747372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 271/552] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:26.753973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7f260-ece8-4044-bd0a-a670e4b3d025 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:26.800988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 282/553] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:26.921143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7f260-ece8-4044-bd0a-a670e4b3d025 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:26.959615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:26.970301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d7044-e976-45ea-b82b-e1b45d8df888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:26.970853 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d7044-e976-45ea-b82b-e1b45d8df888 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:26.971177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d7044-e976-45ea-b82b-e1b45d8df888 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:26.971828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d7044-e976-45ea-b82b-e1b45d8df888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:27.067100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a47939-f43d-4a23-a17e-fa840871c946 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:27.089251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d7044-e976-45ea-b82b-e1b45d8df888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:27.091278 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d7044-e976-45ea-b82b-e1b45d8df888 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:27.194107 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a47939-f43d-4a23-a17e-fa840871c946 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:27.266239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 272/554] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:27.267496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d7044-e976-45ea-b82b-e1b45d8df888 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:27.269599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86f2d0-9589-49c6-95ec-5ace2be4a87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:27.270266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86f2d0-9589-49c6-95ec-5ace2be4a87b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:27.270550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86f2d0-9589-49c6-95ec-5ace2be4a87b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:27.270823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86f2d0-9589-49c6-95ec-5ace2be4a87b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:27.288136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86f2d0-9589-49c6-95ec-5ace2be4a87b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-3f86f2d0-9589-49c6-95ec-5ace2be4a87b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:27.294060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:27.295026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:27.295842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:27.299471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 283/555] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:27.300356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d420fc8-585d-4b7f-b97a-12091946dd65 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:27.300639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d420fc8-585d-4b7f-b97a-12091946dd65 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:27.300917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d420fc8-585d-4b7f-b97a-12091946dd65 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:27.302729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:27.306874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 284/556] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:27.308842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d420fc8-585d-4b7f-b97a-12091946dd65 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:27.312409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:27.384258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d7044-e976-45ea-b82b-e1b45d8df888 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:27.422761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cd361-f0be-48b6-8819-cf211b0a0b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:27.423859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cd361-f0be-48b6-8819-cf211b0a0b26 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:27.424135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cd361-f0be-48b6-8819-cf211b0a0b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:27.424560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cd361-f0be-48b6-8819-cf211b0a0b26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:27.436563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960cd361-f0be-48b6-8819-cf211b0a0b26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:27.437201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d420fc8-585d-4b7f-b97a-12091946dd65 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:27.438036 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960cd361-f0be-48b6-8819-cf211b0a0b26 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-960cd361-f0be-48b6-8819-cf211b0a0b26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:27.453760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:27.454858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:27.455771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:27.455771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:27.455771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:27.455771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 273/557] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:27.456336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c165c8-b243-489e-b700-d0f54292397d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:27.456336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c165c8-b243-489e-b700-d0f54292397d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:27.456565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c165c8-b243-489e-b700-d0f54292397d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:27.476556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a3c165c8-b243-489e-b700-d0f54292397d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:27.477751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a3c165c8-b243-489e-b700-d0f54292397d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:27.486076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d76499-d1d0-4987-b75e-821c875ca2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:27.486978 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d76499-d1d0-4987-b75e-821c875ca2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:27.487404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d76499-d1d0-4987-b75e-821c875ca2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:27.487820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d76499-d1d0-4987-b75e-821c875ca2d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:27.488634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d76499-d1d0-4987-b75e-821c875ca2d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:27.488939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d76499-d1d0-4987-b75e-821c875ca2d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:27.490311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d76499-d1d0-4987-b75e-821c875ca2d1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:27.499908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 285/558] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:27.503737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d76499-d1d0-4987-b75e-821c875ca2d1 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:27.553927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 274/559] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:27.555161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c165c8-b243-489e-b700-d0f54292397d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:27.578020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:27.584993 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:27.623367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d76499-d1d0-4987-b75e-821c875ca2d1 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:27.666460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8801f14-8dc8-4be2-bbf8-126b39f10420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:27.667290 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801f14-8dc8-4be2-bbf8-126b39f10420 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:27.668204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801f14-8dc8-4be2-bbf8-126b39f10420 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:27.668632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801f14-8dc8-4be2-bbf8-126b39f10420 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:27.681626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8801f14-8dc8-4be2-bbf8-126b39f10420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:27.687371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c165c8-b243-489e-b700-d0f54292397d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-f8801f14-8dc8-4be2-bbf8-126b39f10420 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:27.688889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:27.689850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:27.690702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:27.702979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 286/560] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:27.702979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8801f14-8dc8-4be2-bbf8-126b39f10420 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:27.750464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28e933-f1ff-45ac-b3eb-194684f09378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:27.751150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28e933-f1ff-45ac-b3eb-194684f09378 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:27.751434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28e933-f1ff-45ac-b3eb-194684f09378 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:27.751706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28e933-f1ff-45ac-b3eb-194684f09378 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:27.752434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28e933-f1ff-45ac-b3eb-194684f09378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:27.752740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28e933-f1ff-45ac-b3eb-194684f09378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:27.754269 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28e933-f1ff-45ac-b3eb-194684f09378 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:27.771275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 275/561] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:27.772600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28e933-f1ff-45ac-b3eb-194684f09378 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:27.838542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8801f14-8dc8-4be2-bbf8-126b39f10420 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:27.865266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:27.870629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:27.877820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b2e96-2823-4ddb-bd4a-b94f60c5caab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:27.878452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b2e96-2823-4ddb-bd4a-b94f60c5caab None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:27.878728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b2e96-2823-4ddb-bd4a-b94f60c5caab None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:27.879021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b2e96-2823-4ddb-bd4a-b94f60c5caab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:27.891230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b2e96-2823-4ddb-bd4a-b94f60c5caab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:27.892690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b2e96-2823-4ddb-bd4a-b94f60c5caab None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:27.906156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28e933-f1ff-45ac-b3eb-194684f09378 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-319b2e96-2823-4ddb-bd4a-b94f60c5caab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:27.907892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:27.909060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:27.910021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:27.910021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:27.910021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:27.910021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 287/562] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:27.910021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b2e96-2823-4ddb-bd4a-b94f60c5caab None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:27.946847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7b4e3-e691-40d5-96c8-17561196f935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:27.947748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7b4e3-e691-40d5-96c8-17561196f935 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:27.947748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7b4e3-e691-40d5-96c8-17561196f935 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:27.948074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7b4e3-e691-40d5-96c8-17561196f935 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:27.948629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7b4e3-e691-40d5-96c8-17561196f935 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:27.948925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7b4e3-e691-40d5-96c8-17561196f935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:27.950304 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7b4e3-e691-40d5-96c8-17561196f935 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:27.958387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 276/563] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:27.959207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7b4e3-e691-40d5-96c8-17561196f935 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:28.013579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b2e96-2823-4ddb-bd4a-b94f60c5caab None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:28.048083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bc4dc-e30d-45a1-9849-aae2e19c141e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:28.048956 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bc4dc-e30d-45a1-9849-aae2e19c141e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.049233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bc4dc-e30d-45a1-9849-aae2e19c141e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.049506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bc4dc-e30d-45a1-9849-aae2e19c141e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.058011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bc4dc-e30d-45a1-9849-aae2e19c141e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:28.059671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bc4dc-e30d-45a1-9849-aae2e19c141e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:28.068946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7b4e3-e691-40d5-96c8-17561196f935 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-c86bc4dc-e30d-45a1-9849-aae2e19c141e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:28.070920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:28.072163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:28.073174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:28.073174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:28.073174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:28.073174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:28.073174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:28.073174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:28.073174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:28.073174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:28.073174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 288/564] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:28.073174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d25ae-3262-4ed4-bc0d-b0a0e3185186 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.073931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d25ae-3262-4ed4-bc0d-b0a0e3185186 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.073931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d25ae-3262-4ed4-bc0d-b0a0e3185186 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.079072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 289/565] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:28.079072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d25ae-3262-4ed4-bc0d-b0a0e3185186 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:28.103591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:28.109193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff58ed-f648-43b2-aa53-a514296b1809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:28.109824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff58ed-f648-43b2-aa53-a514296b1809 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.110168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff58ed-f648-43b2-aa53-a514296b1809 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.110443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff58ed-f648-43b2-aa53-a514296b1809 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.115582 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:28.121594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff58ed-f648-43b2-aa53-a514296b1809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-76ff58ed-f648-43b2-aa53-a514296b1809 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:28.127673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:28.128733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 277/566] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:28.129562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff58ed-f648-43b2-aa53-a514296b1809 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:28.188025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d25ae-3262-4ed4-bc0d-b0a0e3185186 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:28.235090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d1f0-0dda-4e5c-8409-88659cab8e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:28.236035 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d1f0-0dda-4e5c-8409-88659cab8e1d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.236486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d1f0-0dda-4e5c-8409-88659cab8e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.236936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d1f0-0dda-4e5c-8409-88659cab8e1d None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.238678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86d1f0-0dda-4e5c-8409-88659cab8e1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:28.239618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86d1f0-0dda-4e5c-8409-88659cab8e1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:28.242301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86d1f0-0dda-4e5c-8409-88659cab8e1d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:28.247741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff58ed-f648-43b2-aa53-a514296b1809 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:28.264415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 290/567] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:28.270667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d1f0-0dda-4e5c-8409-88659cab8e1d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:28.293455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd85c6f-faae-4a45-b132-bb0116f719b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:28.294448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd85c6f-faae-4a45-b132-bb0116f719b2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.294448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd85c6f-faae-4a45-b132-bb0116f719b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.294647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd85c6f-faae-4a45-b132-bb0116f719b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.306059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd85c6f-faae-4a45-b132-bb0116f719b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:28.307583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd85c6f-faae-4a45-b132-bb0116f719b2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-fcd85c6f-faae-4a45-b132-bb0116f719b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:28.322551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:28.323618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:28.325007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:28.325007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:28.325007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:28.325007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 278/568] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:28.326013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224af6ed-3ee5-4217-a07d-459ce54789fe None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.326291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224af6ed-3ee5-4217-a07d-459ce54789fe None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.326561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224af6ed-3ee5-4217-a07d-459ce54789fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.346459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-224af6ed-3ee5-4217-a07d-459ce54789fe None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:28.347792 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-224af6ed-3ee5-4217-a07d-459ce54789fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:28.388095 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:28.394567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d1f0-0dda-4e5c-8409-88659cab8e1d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:28.399902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:28.429291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 279/569] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:28.430285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224af6ed-3ee5-4217-a07d-459ce54789fe None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:28.444847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5892e06-9dc0-44e1-81ca-bd92d75d41ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:28.445746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5892e06-9dc0-44e1-81ca-bd92d75d41ba None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.446032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5892e06-9dc0-44e1-81ca-bd92d75d41ba None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.446828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5892e06-9dc0-44e1-81ca-bd92d75d41ba None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.447848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5892e06-9dc0-44e1-81ca-bd92d75d41ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:28.448163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5892e06-9dc0-44e1-81ca-bd92d75d41ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:28.450293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5892e06-9dc0-44e1-81ca-bd92d75d41ba None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:28.471366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 291/570] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:28.472321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5892e06-9dc0-44e1-81ca-bd92d75d41ba None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:28.554886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224af6ed-3ee5-4217-a07d-459ce54789fe None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:28.590733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920393d-8074-4bfb-9a68-bffae888618f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:28.591371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920393d-8074-4bfb-9a68-bffae888618f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.591649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920393d-8074-4bfb-9a68-bffae888618f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.591918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920393d-8074-4bfb-9a68-bffae888618f None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.592579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920393d-8074-4bfb-9a68-bffae888618f None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:28.592879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920393d-8074-4bfb-9a68-bffae888618f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:28.594234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920393d-8074-4bfb-9a68-bffae888618f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:28.601389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5892e06-9dc0-44e1-81ca-bd92d75d41ba None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:28.606752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 280/571] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:28.607435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920393d-8074-4bfb-9a68-bffae888618f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:28.639096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357737f-a67c-49ae-a689-e9768f1eaa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:28.639766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357737f-a67c-49ae-a689-e9768f1eaa2c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.640040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357737f-a67c-49ae-a689-e9768f1eaa2c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.640305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357737f-a67c-49ae-a689-e9768f1eaa2c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.643399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:28.650144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357737f-a67c-49ae-a689-e9768f1eaa2c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:28.652407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357737f-a67c-49ae-a689-e9768f1eaa2c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e357737f-a67c-49ae-a689-e9768f1eaa2c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:28.663793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:28.664751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:28.665736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:28.665736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:28.665736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:28.665736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:28.665736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:28.665736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:28.665736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:28.665736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:28.665736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:28.665736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:28.669635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 292/572] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 22 16:53:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:28.670602 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357737f-a67c-49ae-a689-e9768f1eaa2c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:28.695524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:28.719523 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920393d-8074-4bfb-9a68-bffae888618f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:28.754656 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d3961-3a4b-4834-9aff-81c7787d6d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:28.755301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3961-3a4b-4834-9aff-81c7787d6d34 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.755579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3961-3a4b-4834-9aff-81c7787d6d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.755850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3961-3a4b-4834-9aff-81c7787d6d34 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.756486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d3961-3a4b-4834-9aff-81c7787d6d34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:28.756786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d3961-3a4b-4834-9aff-81c7787d6d34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:28.758177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d3961-3a4b-4834-9aff-81c7787d6d34 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:28.770708 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 281/573] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:28.771639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d3961-3a4b-4834-9aff-81c7787d6d34 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:28.787712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357737f-a67c-49ae-a689-e9768f1eaa2c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:28.825571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d886ab8-55bb-46bf-a91c-68e887f2a42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:28.826170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d886ab8-55bb-46bf-a91c-68e887f2a42d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.826445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d886ab8-55bb-46bf-a91c-68e887f2a42d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.826714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d886ab8-55bb-46bf-a91c-68e887f2a42d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.827459 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d886ab8-55bb-46bf-a91c-68e887f2a42d None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=b1a54beb68ba4610976524684bc26d5b) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:28.827762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d886ab8-55bb-46bf-a91c-68e887f2a42d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:28.829219 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d886ab8-55bb-46bf-a91c-68e887f2a42d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:28.834502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 293/574] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:28 2022] GET /identity/v3/role_assignments?user.id=b1a54beb68ba4610976524684bc26d5b&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:28.835318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d886ab8-55bb-46bf-a91c-68e887f2a42d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:28.880884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:28.889296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d3961-3a4b-4834-9aff-81c7787d6d34 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:28.929825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd271f-b6dd-41db-ab1c-00d615b89b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:28.930900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd271f-b6dd-41db-ab1c-00d615b89b0c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:28.931245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd271f-b6dd-41db-ab1c-00d615b89b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:28.931519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd271f-b6dd-41db-ab1c-00d615b89b0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:28.934619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:28.944312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd271f-b6dd-41db-ab1c-00d615b89b0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:28.945825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd271f-b6dd-41db-ab1c-00d615b89b0c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:28.957291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d886ab8-55bb-46bf-a91c-68e887f2a42d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-5edd271f-b6dd-41db-ab1c-00d615b89b0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:28.963187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:28.964946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:28.965957 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:28.965957 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:28.965957 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:28.965957 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 282/575] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:28.967276 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd271f-b6dd-41db-ab1c-00d615b89b0c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:28.999889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0450cd-4a85-497e-bef4-56a6c8690c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:29.000498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0450cd-4a85-497e-bef4-56a6c8690c80 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:29.000778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0450cd-4a85-497e-bef4-56a6c8690c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:29.001042 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0450cd-4a85-497e-bef4-56a6c8690c80 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:29.001768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0450cd-4a85-497e-bef4-56a6c8690c80 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:29.002073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0450cd-4a85-497e-bef4-56a6c8690c80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:29.003526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0450cd-4a85-497e-bef4-56a6c8690c80 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:29.027295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 294/576] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 22 16:53:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:29.028150 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0450cd-4a85-497e-bef4-56a6c8690c80 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:29.085770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd271f-b6dd-41db-ab1c-00d615b89b0c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:29.123130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496d36a-3b21-4e7e-9251-528222c87522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:29.123842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496d36a-3b21-4e7e-9251-528222c87522 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:29.124119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496d36a-3b21-4e7e-9251-528222c87522 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:29.124490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496d36a-3b21-4e7e-9251-528222c87522 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:29.135540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496d36a-3b21-4e7e-9251-528222c87522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:29.140502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0450cd-4a85-497e-bef4-56a6c8690c80 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-e496d36a-3b21-4e7e-9251-528222c87522 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:29.141168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:29.142090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:29.142982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 283/577] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:29.143711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496d36a-3b21-4e7e-9251-528222c87522 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:29.154997 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:29.184737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e1b37-7b9d-4135-b02e-d0bd98c45b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:29.185737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e1b37-7b9d-4135-b02e-d0bd98c45b00 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:29.186674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e1b37-7b9d-4135-b02e-d0bd98c45b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:29.187185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e1b37-7b9d-4135-b02e-d0bd98c45b00 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:29.196968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e1b37-7b9d-4135-b02e-d0bd98c45b00 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:29.198751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e1b37-7b9d-4135-b02e-d0bd98c45b00 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:29.201228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-bb7e1b37-7b9d-4135-b02e-d0bd98c45b00 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:29.212134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:29.214320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:29.215874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:29.215874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:29.215874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:29.215874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:29.215874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:29.215874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:29.215874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:29.215874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:29.215874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 16:53:29.215874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:29.223235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 295/578] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:29.223967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e1b37-7b9d-4135-b02e-d0bd98c45b00 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:29.260401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496d36a-3b21-4e7e-9251-528222c87522 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:29.297981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ec272-0e5e-4e8e-88fa-9f0b6aec0503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:29.298859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ec272-0e5e-4e8e-88fa-9f0b6aec0503 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:29.299175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ec272-0e5e-4e8e-88fa-9f0b6aec0503 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:29.299443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ec272-0e5e-4e8e-88fa-9f0b6aec0503 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:29.300128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ec272-0e5e-4e8e-88fa-9f0b6aec0503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:29.300451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ec272-0e5e-4e8e-88fa-9f0b6aec0503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:29.302111 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ec272-0e5e-4e8e-88fa-9f0b6aec0503 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:29.319631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 284/579] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:29.320650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ec272-0e5e-4e8e-88fa-9f0b6aec0503 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:29.333533 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e1b37-7b9d-4135-b02e-d0bd98c45b00 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:29.384081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db72718-75f4-4c92-b534-227357852a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:29.384724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db72718-75f4-4c92-b534-227357852a09 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:29.384992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db72718-75f4-4c92-b534-227357852a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:29.385266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db72718-75f4-4c92-b534-227357852a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:29.395825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:29.396552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db72718-75f4-4c92-b534-227357852a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:29.398007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db72718-75f4-4c92-b534-227357852a09 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-1db72718-75f4-4c92-b534-227357852a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:29.413456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:29.414503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:29.415429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:29.415429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:29.415429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:29.415429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 296/580] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:29.415429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db72718-75f4-4c92-b534-227357852a09 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:29.440320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ec272-0e5e-4e8e-88fa-9f0b6aec0503 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:29.472450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:29.475650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dee633-c6dd-4f50-9caf-3761f61e6416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:29.476275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dee633-c6dd-4f50-9caf-3761f61e6416 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:29.476554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dee633-c6dd-4f50-9caf-3761f61e6416 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:29.476821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dee633-c6dd-4f50-9caf-3761f61e6416 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:29.477513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dee633-c6dd-4f50-9caf-3761f61e6416 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:29.477809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dee633-c6dd-4f50-9caf-3761f61e6416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:29.479253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dee633-c6dd-4f50-9caf-3761f61e6416 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:29.492761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 285/581] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:29.494329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dee633-c6dd-4f50-9caf-3761f61e6416 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:29.540769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db72718-75f4-4c92-b534-227357852a09 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:29.580249 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858dd20e-ac1d-4a7c-96d0-9f8cab4994e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:29.581125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dd20e-ac1d-4a7c-96d0-9f8cab4994e2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:29.581567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dd20e-ac1d-4a7c-96d0-9f8cab4994e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:29.581992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dd20e-ac1d-4a7c-96d0-9f8cab4994e2 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:29.582873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858dd20e-ac1d-4a7c-96d0-9f8cab4994e2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:29.583361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858dd20e-ac1d-4a7c-96d0-9f8cab4994e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:29.585063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858dd20e-ac1d-4a7c-96d0-9f8cab4994e2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:29.604703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 297/582] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 22 16:53:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:29.606436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858dd20e-ac1d-4a7c-96d0-9f8cab4994e2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:29.614092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dee633-c6dd-4f50-9caf-3761f61e6416 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:29.643105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:29.654808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab9fb3-06f1-4cf8-bafa-60f73f1ea956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:29.655457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab9fb3-06f1-4cf8-bafa-60f73f1ea956 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:29.655739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab9fb3-06f1-4cf8-bafa-60f73f1ea956 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:29.656057 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab9fb3-06f1-4cf8-bafa-60f73f1ea956 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:29.664446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab9fb3-06f1-4cf8-bafa-60f73f1ea956 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:29.665924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab9fb3-06f1-4cf8-bafa-60f73f1ea956 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-6aab9fb3-06f1-4cf8-bafa-60f73f1ea956 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:29.678498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:29.679647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 286/583] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:29.680611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab9fb3-06f1-4cf8-bafa-60f73f1ea956 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:29.742109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858dd20e-ac1d-4a7c-96d0-9f8cab4994e2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:29.755884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:29.790663 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6b9a0-0fee-47cb-80de-354875812103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:29.791302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6b9a0-0fee-47cb-80de-354875812103 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:29.791578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6b9a0-0fee-47cb-80de-354875812103 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:29.791847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6b9a0-0fee-47cb-80de-354875812103 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:29.792716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6b9a0-0fee-47cb-80de-354875812103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:29.792831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6b9a0-0fee-47cb-80de-354875812103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:29.794162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6b9a0-0fee-47cb-80de-354875812103 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:29.806454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab9fb3-06f1-4cf8-bafa-60f73f1ea956 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:29.811565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 298/584] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:29.811977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6b9a0-0fee-47cb-80de-354875812103 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:29.852263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2ca8d-5056-4ae1-9284-c7a93cb578ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:29.852895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2ca8d-5056-4ae1-9284-c7a93cb578ac None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:29.853170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2ca8d-5056-4ae1-9284-c7a93cb578ac None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:29.853437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2ca8d-5056-4ae1-9284-c7a93cb578ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:29.865516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2ca8d-5056-4ae1-9284-c7a93cb578ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:29.867002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2ca8d-5056-4ae1-9284-c7a93cb578ac None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-b9b2ca8d-5056-4ae1-9284-c7a93cb578ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:29.881594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:29.882565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:29.883833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:29.883833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:29.883833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:29.883833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 287/585] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:29.883833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2ca8d-5056-4ae1-9284-c7a93cb578ac None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:29.926930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6b9a0-0fee-47cb-80de-354875812103 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:29.934088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:29.965456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f998a97-5833-443a-886c-8050c843367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:29.966100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f998a97-5833-443a-886c-8050c843367f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:29.966375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f998a97-5833-443a-886c-8050c843367f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:29.966650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f998a97-5833-443a-886c-8050c843367f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:29.979441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f998a97-5833-443a-886c-8050c843367f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:29.981137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f998a97-5833-443a-886c-8050c843367f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:29.996447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2ca8d-5056-4ae1-9284-c7a93cb578ac None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-9f998a97-5833-443a-886c-8050c843367f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:29.997241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:29.998155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:29.999671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:29.999671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:29.999671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:29.999671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 299/586] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:29.999671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f998a97-5833-443a-886c-8050c843367f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:30.027495 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:30.038933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa72c0-e832-4aa7-a09e-b04180472985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:30.039566 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa72c0-e832-4aa7-a09e-b04180472985 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:30.039842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa72c0-e832-4aa7-a09e-b04180472985 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:30.040108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa72c0-e832-4aa7-a09e-b04180472985 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:30.040831 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa72c0-e832-4aa7-a09e-b04180472985 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:30.041133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa72c0-e832-4aa7-a09e-b04180472985 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:30.042560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa72c0-e832-4aa7-a09e-b04180472985 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:30.062984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 288/587] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 22 16:53:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:30.062984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa72c0-e832-4aa7-a09e-b04180472985 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:30.121613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f998a97-5833-443a-886c-8050c843367f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:30.161106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd5a5c-e594-4355-94f5-638eed84f14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:30.161770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd5a5c-e594-4355-94f5-638eed84f14f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:30.162325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd5a5c-e594-4355-94f5-638eed84f14f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:30.162325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd5a5c-e594-4355-94f5-638eed84f14f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:30.169981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd5a5c-e594-4355-94f5-638eed84f14f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:30.171462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd5a5c-e594-4355-94f5-638eed84f14f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:30.173118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:30.177010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa72c0-e832-4aa7-a09e-b04180472985 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b8bd5a5c-e594-4355-94f5-638eed84f14f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:30.184466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:30.185691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:30.186608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:30.186608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:30.186608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:30.186608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:30.186608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:30.186608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:30.186608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:30.186608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:30.190480 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 300/588] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:30.191420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd5a5c-e594-4355-94f5-638eed84f14f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:30.218292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a4e1a-7be8-4a24-876e-da37dd3aafc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:30.219046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a4e1a-7be8-4a24-876e-da37dd3aafc2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:30.219332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a4e1a-7be8-4a24-876e-da37dd3aafc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:30.219605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a4e1a-7be8-4a24-876e-da37dd3aafc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:30.220243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a4e1a-7be8-4a24-876e-da37dd3aafc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:30.220547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a4e1a-7be8-4a24-876e-da37dd3aafc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:30.222148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a4e1a-7be8-4a24-876e-da37dd3aafc2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:30.235926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 289/589] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:30.239905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a4e1a-7be8-4a24-876e-da37dd3aafc2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:30.283378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:30.304704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd5a5c-e594-4355-94f5-638eed84f14f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:30.346576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea02776-f02e-4182-86b5-876f1491b256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:30.347288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea02776-f02e-4182-86b5-876f1491b256 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:30.347590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea02776-f02e-4182-86b5-876f1491b256 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:30.347929 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea02776-f02e-4182-86b5-876f1491b256 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:30.348579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea02776-f02e-4182-86b5-876f1491b256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:30.348886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea02776-f02e-4182-86b5-876f1491b256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:30.350265 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea02776-f02e-4182-86b5-876f1491b256 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:30.361630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a4e1a-7be8-4a24-876e-da37dd3aafc2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:30.362576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 301/590] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:30.363534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea02776-f02e-4182-86b5-876f1491b256 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:30.403262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019be0a-7957-4c71-bfe9-947640da9280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:30.404412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019be0a-7957-4c71-bfe9-947640da9280 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:30.404412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019be0a-7957-4c71-bfe9-947640da9280 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:30.404412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019be0a-7957-4c71-bfe9-947640da9280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:30.416039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019be0a-7957-4c71-bfe9-947640da9280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:30.418062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019be0a-7957-4c71-bfe9-947640da9280 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-a019be0a-7957-4c71-bfe9-947640da9280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:30.433932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:30.435144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:30.436217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:30.436217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:30.436217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:30.439959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 290/591] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:30.440722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019be0a-7957-4c71-bfe9-947640da9280 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:30.456673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:30.478324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea02776-f02e-4182-86b5-876f1491b256 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:30.518232 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee23c5-34ef-4801-b110-97031cad6fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:30.519123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee23c5-34ef-4801-b110-97031cad6fae None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:30.519399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee23c5-34ef-4801-b110-97031cad6fae None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:30.519665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee23c5-34ef-4801-b110-97031cad6fae None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:30.520352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee23c5-34ef-4801-b110-97031cad6fae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:30.520649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee23c5-34ef-4801-b110-97031cad6fae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:30.522075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee23c5-34ef-4801-b110-97031cad6fae None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:30.531166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:30.534833 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 302/592] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:30.535913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee23c5-34ef-4801-b110-97031cad6fae None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:30.557694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019be0a-7957-4c71-bfe9-947640da9280 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:30.595624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f8984-f8d1-4de8-b139-7626e679cdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:30.596253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f8984-f8d1-4de8-b139-7626e679cdea None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:30.596533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f8984-f8d1-4de8-b139-7626e679cdea None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:30.596930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f8984-f8d1-4de8-b139-7626e679cdea None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:30.606244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f8984-f8d1-4de8-b139-7626e679cdea None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:30.607747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f8984-f8d1-4de8-b139-7626e679cdea None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-ed1f8984-f8d1-4de8-b139-7626e679cdea None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:30.620222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:30.621469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 291/593] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:30.622410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f8984-f8d1-4de8-b139-7626e679cdea None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:30.652279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee23c5-34ef-4801-b110-97031cad6fae None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:30.690642 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6697e-b965-4dd4-9d0a-be71333cbbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:30.691370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6697e-b965-4dd4-9d0a-be71333cbbba None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:30.691647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6697e-b965-4dd4-9d0a-be71333cbbba None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:30.691925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6697e-b965-4dd4-9d0a-be71333cbbba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:30.700655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6697e-b965-4dd4-9d0a-be71333cbbba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:30.701542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:30.703930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6697e-b965-4dd4-9d0a-be71333cbbba None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-fea6697e-b965-4dd4-9d0a-be71333cbbba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:30.717640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:30.719167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:30.720059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:30.720059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:30.720059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:30.720059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:30.720059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:30.720059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:30.720059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:30.720059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:30.720059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 303/594] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:30.720059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6697e-b965-4dd4-9d0a-be71333cbbba None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:30.742900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f8984-f8d1-4de8-b139-7626e679cdea None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:30.783805 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426e8ad-484b-4514-8742-0610877af382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:30.784406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e8ad-484b-4514-8742-0610877af382 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:30.784681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e8ad-484b-4514-8742-0610877af382 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:30.784947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e8ad-484b-4514-8742-0610877af382 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:30.785580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426e8ad-484b-4514-8742-0610877af382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:30.785874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426e8ad-484b-4514-8742-0610877af382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:30.787255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426e8ad-484b-4514-8742-0610877af382 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:30.788983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:30.804592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 292/595] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:30.804592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426e8ad-484b-4514-8742-0610877af382 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:30.840968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6697e-b965-4dd4-9d0a-be71333cbbba None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:30.877002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf01f80-beda-404f-9d5e-c62851b5db63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:30.877664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf01f80-beda-404f-9d5e-c62851b5db63 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:30.877949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf01f80-beda-404f-9d5e-c62851b5db63 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:30.878227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf01f80-beda-404f-9d5e-c62851b5db63 None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/f6c1ce47d8704111988a78e625fd968c/roles/6ae0df226be44eccbaa70e0140db1c40` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:30.894175 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf01f80-beda-404f-9d5e-c62851b5db63 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=f6c1ce47d8704111988a78e625fd968c, role_id=6ae0df226be44eccbaa70e0140db1c40)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:30.895703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf01f80-beda-404f-9d5e-c62851b5db63 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:30.922981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 304/596] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:53:30 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/f6c1ce47d8704111988a78e625fd968c/roles/6ae0df226be44eccbaa70e0140db1c40 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:53:30.922981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf01f80-beda-404f-9d5e-c62851b5db63 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:30.923342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426e8ad-484b-4514-8742-0610877af382 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:30.955047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:30.962928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b0157-6d22-49d9-95ed-bad6cdec9172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:30.963583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b0157-6d22-49d9-95ed-bad6cdec9172 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:30.963858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b0157-6d22-49d9-95ed-bad6cdec9172 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:30.964122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b0157-6d22-49d9-95ed-bad6cdec9172 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:30.965398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b0157-6d22-49d9-95ed-bad6cdec9172 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:30.965699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b0157-6d22-49d9-95ed-bad6cdec9172 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:30.967173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b0157-6d22-49d9-95ed-bad6cdec9172 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:30.991838 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 293/597] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Sep 22 16:53:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:30.991838 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b0157-6d22-49d9-95ed-bad6cdec9172 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:31.090574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf01f80-beda-404f-9d5e-c62851b5db63 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:31.104929 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:31.108547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b0157-6d22-49d9-95ed-bad6cdec9172 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:31.138570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25f494-8fdd-4b1d-9fc8-b4a33abc7b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:31.139208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25f494-8fdd-4b1d-9fc8-b4a33abc7b44 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:31.139487 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25f494-8fdd-4b1d-9fc8-b4a33abc7b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:31.139808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25f494-8fdd-4b1d-9fc8-b4a33abc7b44 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:31.140513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25f494-8fdd-4b1d-9fc8-b4a33abc7b44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:31.140822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25f494-8fdd-4b1d-9fc8-b4a33abc7b44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:31.142274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25f494-8fdd-4b1d-9fc8-b4a33abc7b44 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:31.159004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 305/598] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:31.159004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25f494-8fdd-4b1d-9fc8-b4a33abc7b44 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:31.162019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f4661-26a5-4bf9-9827-f9670d9cd412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:31.162700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f4661-26a5-4bf9-9827-f9670d9cd412 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:31.162993 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f4661-26a5-4bf9-9827-f9670d9cd412 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:31.163271 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f4661-26a5-4bf9-9827-f9670d9cd412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:31.171946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f4661-26a5-4bf9-9827-f9670d9cd412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:31.173424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f4661-26a5-4bf9-9827-f9670d9cd412 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-169f4661-26a5-4bf9-9827-f9670d9cd412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:31.185842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:31.186984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:31.187881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:31.187881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:31.187881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:31.187881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:31.187881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:31.187881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:31.187881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:31.187881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:31.187881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 294/599] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:31.188570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412caa02-7e8f-4b35-971e-89056045c4a5 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:31.188756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412caa02-7e8f-4b35-971e-89056045c4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:31.189036 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412caa02-7e8f-4b35-971e-89056045c4a5 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:31.190397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 295/600] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:31.190927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412caa02-7e8f-4b35-971e-89056045c4a5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:31.254496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:31.309955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25f494-8fdd-4b1d-9fc8-b4a33abc7b44 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:31.314683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412caa02-7e8f-4b35-971e-89056045c4a5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:31.350568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f933f-949d-4523-8723-4a01ef3a9f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:31.351253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f933f-949d-4523-8723-4a01ef3a9f4e None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:31.351531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f933f-949d-4523-8723-4a01ef3a9f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:31.351802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f933f-949d-4523-8723-4a01ef3a9f4e None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/ae1b8a51c9fa4f2594bdd79b58962b0a/roles/6ae0df226be44eccbaa70e0140db1c40` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:31.353544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98be547-8618-4183-a912-217521feec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:31.354605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98be547-8618-4183-a912-217521feec71 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:31.356044 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98be547-8618-4183-a912-217521feec71 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:31.356501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98be547-8618-4183-a912-217521feec71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:31.368535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f933f-949d-4523-8723-4a01ef3a9f4e None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=ae1b8a51c9fa4f2594bdd79b58962b0a, role_id=6ae0df226be44eccbaa70e0140db1c40)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:31.369702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98be547-8618-4183-a912-217521feec71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:31.369977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f933f-949d-4523-8723-4a01ef3a9f4e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:31.371586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98be547-8618-4183-a912-217521feec71 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:31.385253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 296/601] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:53:31 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/ae1b8a51c9fa4f2594bdd79b58962b0a/roles/6ae0df226be44eccbaa70e0140db1c40 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:53:31.386776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f933f-949d-4523-8723-4a01ef3a9f4e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-f98be547-8618-4183-a912-217521feec71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:31.389626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:31.390675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:31.391598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:31.391598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:31.391598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:31.393572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 306/602] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:31.395125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a27ab-b4f8-4b02-883d-8c32fa71127d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:31.395581 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a27ab-b4f8-4b02-883d-8c32fa71127d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:31.396038 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a27ab-b4f8-4b02-883d-8c32fa71127d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:31.417744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c98a27ab-b4f8-4b02-883d-8c32fa71127d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:31.419446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c98a27ab-b4f8-4b02-883d-8c32fa71127d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:31.443333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:31.505214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 307/603] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:31.509017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a27ab-b4f8-4b02-883d-8c32fa71127d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:31.561609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:31.565385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f933f-949d-4523-8723-4a01ef3a9f4e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:31.645035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa35c1-c80e-4604-885c-543bf29e8e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:31.645651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa35c1-c80e-4604-885c-543bf29e8e15 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:31.645926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa35c1-c80e-4604-885c-543bf29e8e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:31.646198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa35c1-c80e-4604-885c-543bf29e8e15 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:31.646895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa35c1-c80e-4604-885c-543bf29e8e15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:31.647224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa35c1-c80e-4604-885c-543bf29e8e15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:31.648659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa35c1-c80e-4604-885c-543bf29e8e15 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:31.677384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a27ab-b4f8-4b02-883d-8c32fa71127d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:31.677842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 297/604] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:31.685789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa35c1-c80e-4604-885c-543bf29e8e15 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:31.722143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb74ac-d1da-40a2-966e-fefdb3c2d50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:31.722985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb74ac-d1da-40a2-966e-fefdb3c2d50c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:31.723347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb74ac-d1da-40a2-966e-fefdb3c2d50c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:31.723626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb74ac-d1da-40a2-966e-fefdb3c2d50c None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:31.724421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb74ac-d1da-40a2-966e-fefdb3c2d50c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:31.725016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb74ac-d1da-40a2-966e-fefdb3c2d50c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:31.726433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb74ac-d1da-40a2-966e-fefdb3c2d50c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:31.740087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 308/605] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:31.746016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb74ac-d1da-40a2-966e-fefdb3c2d50c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:31.770657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:31.875997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb74ac-d1da-40a2-966e-fefdb3c2d50c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:31.904688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa35c1-c80e-4604-885c-543bf29e8e15 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:31.914757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36876218-c7a0-4cf0-ac12-745dbf27d711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:31.915391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36876218-c7a0-4cf0-ac12-745dbf27d711 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:31.915667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36876218-c7a0-4cf0-ac12-745dbf27d711 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:31.915936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36876218-c7a0-4cf0-ac12-745dbf27d711 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:31.925359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36876218-c7a0-4cf0-ac12-745dbf27d711 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:31.926895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36876218-c7a0-4cf0-ac12-745dbf27d711 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-36876218-c7a0-4cf0-ac12-745dbf27d711 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:31.939755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:31.940732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:31.941617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:31.941617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:31.941617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:31.941617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:31.941617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:31.941617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:31.941617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:31.941617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:31.941617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 309/606] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:31.942101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36876218-c7a0-4cf0-ac12-745dbf27d711 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:31.953692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6a1b0-e7c6-43e7-8454-65d421da1cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:31.953692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6a1b0-e7c6-43e7-8454-65d421da1cdb None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:31.953692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6a1b0-e7c6-43e7-8454-65d421da1cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:31.953692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6a1b0-e7c6-43e7-8454-65d421da1cdb None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/6103e9a612c84a9ea7d21cc5128fa40b/roles/6ae0df226be44eccbaa70e0140db1c40` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:31.969333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6a1b0-e7c6-43e7-8454-65d421da1cdb None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=6103e9a612c84a9ea7d21cc5128fa40b, role_id=6ae0df226be44eccbaa70e0140db1c40)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:31.971237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6a1b0-e7c6-43e7-8454-65d421da1cdb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:31.988229 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 298/607] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:53:31 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/6103e9a612c84a9ea7d21cc5128fa40b/roles/6ae0df226be44eccbaa70e0140db1c40 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:53:31.989082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6a1b0-e7c6-43e7-8454-65d421da1cdb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:32.021541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:32.036235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:32.074226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36876218-c7a0-4cf0-ac12-745dbf27d711 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:32.114366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0472ef-7ce0-4c13-8f3c-412224282d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:32.115341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0472ef-7ce0-4c13-8f3c-412224282d96 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:32.115797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0472ef-7ce0-4c13-8f3c-412224282d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:32.116275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0472ef-7ce0-4c13-8f3c-412224282d96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:32.129746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0472ef-7ce0-4c13-8f3c-412224282d96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-ca0472ef-7ce0-4c13-8f3c-412224282d96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:32.135715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:32.137315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:32.138281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:32.145604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 310/608] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:32.147272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0472ef-7ce0-4c13-8f3c-412224282d96 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:32.234830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6a1b0-e7c6-43e7-8454-65d421da1cdb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:32.286328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd508bc-02df-4cda-a9a3-b7265359f431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:32.286877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd508bc-02df-4cda-a9a3-b7265359f431 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:32.287183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd508bc-02df-4cda-a9a3-b7265359f431 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:32.287458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd508bc-02df-4cda-a9a3-b7265359f431 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:32.288163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd508bc-02df-4cda-a9a3-b7265359f431 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:32.288474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd508bc-02df-4cda-a9a3-b7265359f431 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:32.289926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd508bc-02df-4cda-a9a3-b7265359f431 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:32.293415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0472ef-7ce0-4c13-8f3c-412224282d96 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:32.309588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 299/609] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:32.318550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd508bc-02df-4cda-a9a3-b7265359f431 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:32.358282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:32.371052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00b948-6b78-4924-bb86-eaca8c0f47a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:32.371684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00b948-6b78-4924-bb86-eaca8c0f47a5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:32.371963 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00b948-6b78-4924-bb86-eaca8c0f47a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:32.372237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00b948-6b78-4924-bb86-eaca8c0f47a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:32.372913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00b948-6b78-4924-bb86-eaca8c0f47a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:32.373217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00b948-6b78-4924-bb86-eaca8c0f47a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:32.374986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00b948-6b78-4924-bb86-eaca8c0f47a5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:32.403606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 311/610] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:32.410875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00b948-6b78-4924-bb86-eaca8c0f47a5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:32.470107 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:32.519083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd508bc-02df-4cda-a9a3-b7265359f431 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:32.559323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512ec77-f984-4c5c-9f88-c0602e39bdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:32.559929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512ec77-f984-4c5c-9f88-c0602e39bdba None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:32.560213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512ec77-f984-4c5c-9f88-c0602e39bdba None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:32.560491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512ec77-f984-4c5c-9f88-c0602e39bdba None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/8c2ed1eb664348e384824f39ca90c950/roles/6ae0df226be44eccbaa70e0140db1c40` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:32.578644 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512ec77-f984-4c5c-9f88-c0602e39bdba None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=8c2ed1eb664348e384824f39ca90c950, role_id=6ae0df226be44eccbaa70e0140db1c40)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:32.580263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512ec77-f984-4c5c-9f88-c0602e39bdba None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:32.598789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 300/611] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:53:32 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/8c2ed1eb664348e384824f39ca90c950/roles/6ae0df226be44eccbaa70e0140db1c40 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:53:32.661515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00b948-6b78-4924-bb86-eaca8c0f47a5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:32.744450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5909e76-6222-46ae-bc47-64acaabe18c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:32.746091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909e76-6222-46ae-bc47-64acaabe18c4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:32.746586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909e76-6222-46ae-bc47-64acaabe18c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:32.747082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909e76-6222-46ae-bc47-64acaabe18c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:32.792840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5909e76-6222-46ae-bc47-64acaabe18c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:32.794412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5909e76-6222-46ae-bc47-64acaabe18c4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e5909e76-6222-46ae-bc47-64acaabe18c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:32.835492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:32.836475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:32.837357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:32.837357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:32.837357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:32.838110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 312/612] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:32.844843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512ec77-f984-4c5c-9f88-c0602e39bdba None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:32.985069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512ec77-f984-4c5c-9f88-c0602e39bdba None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:33.019429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:33.044346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f9610-7660-479d-bf2e-867a10ffa76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:33.044975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f9610-7660-479d-bf2e-867a10ffa76f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:33.045257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f9610-7660-479d-bf2e-867a10ffa76f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:33.045533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f9610-7660-479d-bf2e-867a10ffa76f None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:33.046189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f9610-7660-479d-bf2e-867a10ffa76f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:33.046491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f9610-7660-479d-bf2e-867a10ffa76f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:33.047962 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f9610-7660-479d-bf2e-867a10ffa76f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:33.061582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 301/613] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:33.067124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5909e76-6222-46ae-bc47-64acaabe18c4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:33.160750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:33.240538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5909e76-6222-46ae-bc47-64acaabe18c4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:33.293207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eede83-8cca-4b49-a59d-5c56af3f4f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:33.293855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eede83-8cca-4b49-a59d-5c56af3f4f80 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:33.294131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eede83-8cca-4b49-a59d-5c56af3f4f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:33.294439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eede83-8cca-4b49-a59d-5c56af3f4f80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:33.304189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eede83-8cca-4b49-a59d-5c56af3f4f80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:33.305652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eede83-8cca-4b49-a59d-5c56af3f4f80 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-79eede83-8cca-4b49-a59d-5c56af3f4f80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:33.324842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:33.325867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:33.326812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:33.326812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:33.326812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:33.326812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:33.326812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:33.326812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:33.326812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:33.326812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:33.326812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:33.326812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:33.326812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 313/614] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:33.332188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f9610-7660-479d-bf2e-867a10ffa76f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:33.502322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f9610-7660-479d-bf2e-867a10ffa76f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:33.558290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d8513-0f78-4e13-9fd4-1ef8a58be96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:33.559327 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d8513-0f78-4e13-9fd4-1ef8a58be96d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:33.559817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d8513-0f78-4e13-9fd4-1ef8a58be96d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:33.560288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d8513-0f78-4e13-9fd4-1ef8a58be96d None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:33.561258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d8513-0f78-4e13-9fd4-1ef8a58be96d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:33.561767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d8513-0f78-4e13-9fd4-1ef8a58be96d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:33.563533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d8513-0f78-4e13-9fd4-1ef8a58be96d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:33.587733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 302/615] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 22 16:53:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:33.593888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eede83-8cca-4b49-a59d-5c56af3f4f80 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:33.771713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eede83-8cca-4b49-a59d-5c56af3f4f80 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:33.782216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:33.837513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260dd7f-5fbe-400b-b12d-f605b231991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:33.838175 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260dd7f-5fbe-400b-b12d-f605b231991c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:33.838454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260dd7f-5fbe-400b-b12d-f605b231991c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:33.843771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260dd7f-5fbe-400b-b12d-f605b231991c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:33.867581 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260dd7f-5fbe-400b-b12d-f605b231991c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:33.869140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260dd7f-5fbe-400b-b12d-f605b231991c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-7260dd7f-5fbe-400b-b12d-f605b231991c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:33.893632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:33.894666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:33.895612 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:33.895612 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:33.895612 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:33.896225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 314/616] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:33.904084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d8513-0f78-4e13-9fd4-1ef8a58be96d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:33.934006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feb8a4-de33-4822-9b6d-f055c37767e5 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:33.934353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feb8a4-de33-4822-9b6d-f055c37767e5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:33.934574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feb8a4-de33-4822-9b6d-f055c37767e5 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:33.936394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 315/617] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:53:33.951558 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3734da6-7f43-4801-92fa-57563c006d71 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:33.951851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3734da6-7f43-4801-92fa-57563c006d71 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:33.952104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3734da6-7f43-4801-92fa-57563c006d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:33.962540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:33.992009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e3734da6-7f43-4801-92fa-57563c006d71 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:33.993704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e3734da6-7f43-4801-92fa-57563c006d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:34.088346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d8513-0f78-4e13-9fd4-1ef8a58be96d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:34.126980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 316/618] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:34.126980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82e05-1b88-407c-b360-d99df9a11174 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:34.126980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82e05-1b88-407c-b360-d99df9a11174 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:34.126980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82e05-1b88-407c-b360-d99df9a11174 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:34.126980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 317/619] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:34.130030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537f21a-7550-467e-b700-f08c2b168b03 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:34.130314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537f21a-7550-467e-b700-f08c2b168b03 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:34.130585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537f21a-7550-467e-b700-f08c2b168b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:34.150193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f537f21a-7550-467e-b700-f08c2b168b03 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:34.151479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f537f21a-7550-467e-b700-f08c2b168b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:34.171696 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0d3dc-2bb6-465f-bedd-eb36a218746a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:34.172879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0d3dc-2bb6-465f-bedd-eb36a218746a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:34.173369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0d3dc-2bb6-465f-bedd-eb36a218746a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:34.174087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0d3dc-2bb6-465f-bedd-eb36a218746a None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:34.174967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0d3dc-2bb6-465f-bedd-eb36a218746a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:34.175475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0d3dc-2bb6-465f-bedd-eb36a218746a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:34.177102 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0d3dc-2bb6-465f-bedd-eb36a218746a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:34.205423 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 303/620] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:34.208033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0d3dc-2bb6-465f-bedd-eb36a218746a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:34.253230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 318/621] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:34.374397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:34.378932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0d3dc-2bb6-465f-bedd-eb36a218746a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:34.428202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f5f4a-cbf7-48e0-8fb0-a0c68839a006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:34.428847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f5f4a-cbf7-48e0-8fb0-a0c68839a006 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:34.429130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f5f4a-cbf7-48e0-8fb0-a0c68839a006 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:34.429402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f5f4a-cbf7-48e0-8fb0-a0c68839a006 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:34.439218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f5f4a-cbf7-48e0-8fb0-a0c68839a006 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:34.440695 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f5f4a-cbf7-48e0-8fb0-a0c68839a006 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-5e5f5f4a-cbf7-48e0-8fb0-a0c68839a006 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:34.460451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:34.461414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:34.462305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:34.462305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:34.462305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:34.462305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:34.462305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:34.462305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:34.462305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:34.462305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:34.462305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 304/622] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:34.467846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537f21a-7550-467e-b700-f08c2b168b03 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:34.494060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:34.527945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f5f4a-cbf7-48e0-8fb0-a0c68839a006 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:34.610139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537f21a-7550-467e-b700-f08c2b168b03 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:34.657470 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b94c5-03b4-4721-9719-74331379c638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:34.658315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b94c5-03b4-4721-9719-74331379c638 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:34.658756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b94c5-03b4-4721-9719-74331379c638 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:34.659188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f5f4a-cbf7-48e0-8fb0-a0c68839a006 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:34.659354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b94c5-03b4-4721-9719-74331379c638 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:34.660235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b94c5-03b4-4721-9719-74331379c638 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:34.660730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b94c5-03b4-4721-9719-74331379c638 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:34.663587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b94c5-03b4-4721-9719-74331379c638 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:34.682104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 319/623] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:34.683762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b94c5-03b4-4721-9719-74331379c638 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:34.701284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560d77d-0fcb-4e6b-b5d6-1b07451320ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:34.702074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560d77d-0fcb-4e6b-b5d6-1b07451320ef None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:34.702508 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560d77d-0fcb-4e6b-b5d6-1b07451320ef None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:34.702878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560d77d-0fcb-4e6b-b5d6-1b07451320ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:34.715128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8560d77d-0fcb-4e6b-b5d6-1b07451320ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-8560d77d-0fcb-4e6b-b5d6-1b07451320ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:34.720765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:34.721945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:34.723025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:34.729858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 305/624] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:34.731245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560d77d-0fcb-4e6b-b5d6-1b07451320ef None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:34.755821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:34.789191 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:34.813184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b94c5-03b4-4721-9719-74331379c638 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:34.848322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560d77d-0fcb-4e6b-b5d6-1b07451320ef None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:34.852678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f90626-356a-47b7-bfc8-521d3d8dfb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:34.853322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f90626-356a-47b7-bfc8-521d3d8dfb8a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:34.853601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f90626-356a-47b7-bfc8-521d3d8dfb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:34.853871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f90626-356a-47b7-bfc8-521d3d8dfb8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:34.866112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f90626-356a-47b7-bfc8-521d3d8dfb8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b3f90626-356a-47b7-bfc8-521d3d8dfb8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:34.871999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:34.873307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 320/625] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:34.874165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f90626-356a-47b7-bfc8-521d3d8dfb8a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:34.890751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421bfa0-8c4f-4b07-9d67-0151f8d621e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:34.891699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421bfa0-8c4f-4b07-9d67-0151f8d621e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:34.892156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421bfa0-8c4f-4b07-9d67-0151f8d621e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:34.892438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421bfa0-8c4f-4b07-9d67-0151f8d621e5 None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/b1a54beb68ba4610976524684bc26d5b/roles/6ae0df226be44eccbaa70e0140db1c40` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:34.908901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421bfa0-8c4f-4b07-9d67-0151f8d621e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=b1a54beb68ba4610976524684bc26d5b, role_id=6ae0df226be44eccbaa70e0140db1c40)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:34.910503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421bfa0-8c4f-4b07-9d67-0151f8d621e5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:34.935085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 306/626] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:53:34 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/b1a54beb68ba4610976524684bc26d5b/roles/6ae0df226be44eccbaa70e0140db1c40 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:53:34.936657 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd686e6-f83e-4309-9798-783b3d314404 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:34.937000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd686e6-f83e-4309-9798-783b3d314404 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:34.937279 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd686e6-f83e-4309-9798-783b3d314404 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:34.938968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 307/627] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:34.939682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd686e6-f83e-4309-9798-783b3d314404 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:34.997794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f90626-356a-47b7-bfc8-521d3d8dfb8a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:35.039726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f5ec5-d172-483c-bf31-2460556cff85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:35.040793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f5ec5-d172-483c-bf31-2460556cff85 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.041256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f5ec5-d172-483c-bf31-2460556cff85 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.041715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f5ec5-d172-483c-bf31-2460556cff85 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.043074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f5ec5-d172-483c-bf31-2460556cff85 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:35.043481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f5ec5-d172-483c-bf31-2460556cff85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:35.045259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f5ec5-d172-483c-bf31-2460556cff85 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:35.051283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:35.055236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:35.060368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 321/628] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:35.061517 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a9a66-ce9f-4f40-b8b0-82246c027cdd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.062322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a9a66-ce9f-4f40-b8b0-82246c027cdd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.063415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a9a66-ce9f-4f40-b8b0-82246c027cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.084758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e31a9a66-ce9f-4f40-b8b0-82246c027cdd None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:35.086133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e31a9a66-ce9f-4f40-b8b0-82246c027cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:35.102039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd686e6-f83e-4309-9798-783b3d314404 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:35.153145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76bd3-c8de-4160-9131-25cb8bd6ac02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:35.153769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76bd3-c8de-4160-9131-25cb8bd6ac02 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.154049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76bd3-c8de-4160-9131-25cb8bd6ac02 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.154318 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76bd3-c8de-4160-9131-25cb8bd6ac02 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.155461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76bd3-c8de-4160-9131-25cb8bd6ac02 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:35.155461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76bd3-c8de-4160-9131-25cb8bd6ac02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:35.156683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76bd3-c8de-4160-9131-25cb8bd6ac02 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:35.166032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 308/629] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:35.166884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76bd3-c8de-4160-9131-25cb8bd6ac02 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:35.174937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 322/630] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:35.176164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a9a66-ce9f-4f40-b8b0-82246c027cdd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:35.294587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a9a66-ce9f-4f40-b8b0-82246c027cdd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:35.299091 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76bd3-c8de-4160-9131-25cb8bd6ac02 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:35.330260 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:35.339972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec454f-f7f7-4832-8c2c-c8035489a229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:35.340955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec454f-f7f7-4832-8c2c-c8035489a229 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.341239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec454f-f7f7-4832-8c2c-c8035489a229 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.341523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec454f-f7f7-4832-8c2c-c8035489a229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.354561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:35.362470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57615ac6-5b01-4f57-b516-c926549f43ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:35.363156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57615ac6-5b01-4f57-b516-c926549f43ae None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.363440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57615ac6-5b01-4f57-b516-c926549f43ae None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.363800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57615ac6-5b01-4f57-b516-c926549f43ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.365433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec454f-f7f7-4832-8c2c-c8035489a229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:35.367812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec454f-f7f7-4832-8c2c-c8035489a229 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:35.378127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57615ac6-5b01-4f57-b516-c926549f43ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:35.379619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57615ac6-5b01-4f57-b516-c926549f43ae None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-d9ec454f-f7f7-4832-8c2c-c8035489a229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:35.385063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:35.386163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:35.387132 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:35.387132 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:35.387132 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:35.389268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 323/631] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:35.390815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7507-35d2-4610-b5b8-2bb3bfd45243 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.391348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7507-35d2-4610-b5b8-2bb3bfd45243 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.391838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7507-35d2-4610-b5b8-2bb3bfd45243 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-57615ac6-5b01-4f57-b516-c926549f43ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:35.397550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:35.398789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:35.399733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:35.399733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:35.399733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:35.399733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 309/633] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:35.399733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57615ac6-5b01-4f57-b516-c926549f43ae None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:35.400409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 324/633] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:35.403091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fadb8c-a2cc-4f9a-bb72-1cc81f74dff9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.403600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fadb8c-a2cc-4f9a-bb72-1cc81f74dff9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.404085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fadb8c-a2cc-4f9a-bb72-1cc81f74dff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.423371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a5fadb8c-a2cc-4f9a-bb72-1cc81f74dff9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:35.425254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a5fadb8c-a2cc-4f9a-bb72-1cc81f74dff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:35.516275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 325/634] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:35.517006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57615ac6-5b01-4f57-b516-c926549f43ae None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:35.517136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fadb8c-a2cc-4f9a-bb72-1cc81f74dff9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:35.552895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855c140-7c5d-4ecc-b7be-8a20659257a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:35.553501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855c140-7c5d-4ecc-b7be-8a20659257a1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.553843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855c140-7c5d-4ecc-b7be-8a20659257a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.554339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855c140-7c5d-4ecc-b7be-8a20659257a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.555302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855c140-7c5d-4ecc-b7be-8a20659257a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:35.555611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855c140-7c5d-4ecc-b7be-8a20659257a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:35.557061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855c140-7c5d-4ecc-b7be-8a20659257a1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:35.574461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 310/635] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:35.575759 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855c140-7c5d-4ecc-b7be-8a20659257a1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:35.626131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:35.652427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fadb8c-a2cc-4f9a-bb72-1cc81f74dff9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:35.670364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:35.694217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855c140-7c5d-4ecc-b7be-8a20659257a1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:35.702565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b3fda-fcbf-4f2b-8157-72c7e2803340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:35.703467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b3fda-fcbf-4f2b-8157-72c7e2803340 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.704432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b3fda-fcbf-4f2b-8157-72c7e2803340 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.704860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b3fda-fcbf-4f2b-8157-72c7e2803340 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.705666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b3fda-fcbf-4f2b-8157-72c7e2803340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:35.706590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b3fda-fcbf-4f2b-8157-72c7e2803340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:35.708578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b3fda-fcbf-4f2b-8157-72c7e2803340 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:35.731233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 326/636] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:35.731233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b3fda-fcbf-4f2b-8157-72c7e2803340 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:35.742030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f57463-ad07-4f45-80f4-26ec6b600fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:35.743034 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f57463-ad07-4f45-80f4-26ec6b600fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.744006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f57463-ad07-4f45-80f4-26ec6b600fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.744509 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f57463-ad07-4f45-80f4-26ec6b600fa1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.760433 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f57463-ad07-4f45-80f4-26ec6b600fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-43f57463-ad07-4f45-80f4-26ec6b600fa1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:35.773541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:35.774746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:35.775848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:35.778472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 311/637] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:35.779581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f57463-ad07-4f45-80f4-26ec6b600fa1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:35.854099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b3fda-fcbf-4f2b-8157-72c7e2803340 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:35.891513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca509d6-be5b-4eeb-9187-0e4cfe8ead62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:35.892038 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca509d6-be5b-4eeb-9187-0e4cfe8ead62 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.892317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca509d6-be5b-4eeb-9187-0e4cfe8ead62 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.892586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca509d6-be5b-4eeb-9187-0e4cfe8ead62 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.901304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca509d6-be5b-4eeb-9187-0e4cfe8ead62 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:35.903422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca509d6-be5b-4eeb-9187-0e4cfe8ead62 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:35.903780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f57463-ad07-4f45-80f4-26ec6b600fa1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:35.907382 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-5ca509d6-be5b-4eeb-9187-0e4cfe8ead62 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:35.916011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:35.917060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:35.918064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:35.918064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:35.918064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:35.918064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:35.918064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:35.918064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:35.918064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:35.918064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:35.918064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 16:53:35.918064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:35.922050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 327/638] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:35.923355 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca509d6-be5b-4eeb-9187-0e4cfe8ead62 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:35.946265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768e7d-ff26-451e-b3ac-879b17f1f6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:35.947056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768e7d-ff26-451e-b3ac-879b17f1f6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:35.947347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768e7d-ff26-451e-b3ac-879b17f1f6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:35.947652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768e7d-ff26-451e-b3ac-879b17f1f6b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:35.956977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89768e7d-ff26-451e-b3ac-879b17f1f6b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:35.958445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89768e7d-ff26-451e-b3ac-879b17f1f6b8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:35.965422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-89768e7d-ff26-451e-b3ac-879b17f1f6b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:35.970601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:35.971633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 312/639] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 22 16:53:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:35.972562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768e7d-ff26-451e-b3ac-879b17f1f6b8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:36.050586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca509d6-be5b-4eeb-9187-0e4cfe8ead62 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:36.085468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768e7d-ff26-451e-b3ac-879b17f1f6b8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:36.088475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593bdca-19d6-4b11-9712-9d17c8445c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:36.088475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593bdca-19d6-4b11-9712-9d17c8445c5f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:36.088727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593bdca-19d6-4b11-9712-9d17c8445c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:36.088872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593bdca-19d6-4b11-9712-9d17c8445c5f None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:36.090021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593bdca-19d6-4b11-9712-9d17c8445c5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:36.090314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593bdca-19d6-4b11-9712-9d17c8445c5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:36.091699 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593bdca-19d6-4b11-9712-9d17c8445c5f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:36.106128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 328/640] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:36.106128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593bdca-19d6-4b11-9712-9d17c8445c5f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:36.130164 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e00e8-8408-4035-bd74-42d4b17428ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:36.130803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e00e8-8408-4035-bd74-42d4b17428ac None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:36.131125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e00e8-8408-4035-bd74-42d4b17428ac None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:36.131402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e00e8-8408-4035-bd74-42d4b17428ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:36.142381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e00e8-8408-4035-bd74-42d4b17428ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-1e3e00e8-8408-4035-bd74-42d4b17428ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:36.147723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:36.148915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:36.149762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 313/641] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:36.150458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e00e8-8408-4035-bd74-42d4b17428ac None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:36.176454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:36.217934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:36.225440 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593bdca-19d6-4b11-9712-9d17c8445c5f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:36.257227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e00e8-8408-4035-bd74-42d4b17428ac None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:36.260996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e3047-5634-40d6-acfa-6ef105bf34db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:36.261600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e3047-5634-40d6-acfa-6ef105bf34db None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:36.261872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e3047-5634-40d6-acfa-6ef105bf34db None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:36.262136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e3047-5634-40d6-acfa-6ef105bf34db None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:36.262856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e3047-5634-40d6-acfa-6ef105bf34db None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:36.263184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e3047-5634-40d6-acfa-6ef105bf34db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:36.264708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e3047-5634-40d6-acfa-6ef105bf34db None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:36.289156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 329/642] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 22 16:53:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:36.290051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e3047-5634-40d6-acfa-6ef105bf34db None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:36.291777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754da69-6460-4d74-a703-1e2d00590645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:36.292965 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754da69-6460-4d74-a703-1e2d00590645 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:36.293256 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754da69-6460-4d74-a703-1e2d00590645 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:36.293527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754da69-6460-4d74-a703-1e2d00590645 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:36.294292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754da69-6460-4d74-a703-1e2d00590645 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:36.294596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754da69-6460-4d74-a703-1e2d00590645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:36.296092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754da69-6460-4d74-a703-1e2d00590645 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:36.314171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 314/643] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 22 16:53:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:36.315015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754da69-6460-4d74-a703-1e2d00590645 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:36.396793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:36.410124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e3047-5634-40d6-acfa-6ef105bf34db None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:36.416985 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754da69-6460-4d74-a703-1e2d00590645 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:36.451955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72382b96-48f0-4e23-ba68-fac2a41d7ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:36.452897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382b96-48f0-4e23-ba68-fac2a41d7ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:36.453340 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382b96-48f0-4e23-ba68-fac2a41d7ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:36.453453 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219701bb-7075-4a29-9ff1-2a39bbd7b418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:36.453855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382b96-48f0-4e23-ba68-fac2a41d7ee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:36.453964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219701bb-7075-4a29-9ff1-2a39bbd7b418 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:36.454223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219701bb-7075-4a29-9ff1-2a39bbd7b418 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:36.454496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219701bb-7075-4a29-9ff1-2a39bbd7b418 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:36.455187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219701bb-7075-4a29-9ff1-2a39bbd7b418 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:36.455492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219701bb-7075-4a29-9ff1-2a39bbd7b418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:36.456861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219701bb-7075-4a29-9ff1-2a39bbd7b418 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:36.464795 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 315/644] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:36.465660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219701bb-7075-4a29-9ff1-2a39bbd7b418 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:36.470989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72382b96-48f0-4e23-ba68-fac2a41d7ee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:36.472640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72382b96-48f0-4e23-ba68-fac2a41d7ee2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-72382b96-48f0-4e23-ba68-fac2a41d7ee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:36.490543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:36.491655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:36.492684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:36.492684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:36.492684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:36.492684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 330/645] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:36.492905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72382b96-48f0-4e23-ba68-fac2a41d7ee2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:36.498629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:36.571877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219701bb-7075-4a29-9ff1-2a39bbd7b418 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:36.605707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72382b96-48f0-4e23-ba68-fac2a41d7ee2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:36.607161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3110a7-64f9-4d3c-bcb3-285450cb6133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:36.607865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3110a7-64f9-4d3c-bcb3-285450cb6133 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:36.608147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3110a7-64f9-4d3c-bcb3-285450cb6133 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:36.608426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3110a7-64f9-4d3c-bcb3-285450cb6133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:36.620233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:36.646620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934c5f0-d460-4062-8b0b-84d00c97b4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:36.647519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934c5f0-d460-4062-8b0b-84d00c97b4de None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:36.648011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934c5f0-d460-4062-8b0b-84d00c97b4de None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:36.648462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934c5f0-d460-4062-8b0b-84d00c97b4de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:36.678098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3110a7-64f9-4d3c-bcb3-285450cb6133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:36.679599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3110a7-64f9-4d3c-bcb3-285450cb6133 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:36.723357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934c5f0-d460-4062-8b0b-84d00c97b4de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:36.724732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934c5f0-d460-4062-8b0b-84d00c97b4de None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:36.728650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:36.799216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 316/646] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:36.804317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3110a7-64f9-4d3c-bcb3-285450cb6133 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:36.845092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 331/647] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:36.845948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934c5f0-d460-4062-8b0b-84d00c97b4de None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:36.855815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:36.920475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3110a7-64f9-4d3c-bcb3-285450cb6133 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:36.949515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934c5f0-d460-4062-8b0b-84d00c97b4de None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:36.951848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:36.960400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d92c2f-9379-43dd-8cff-19166c14b015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:36.961349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d92c2f-9379-43dd-8cff-19166c14b015 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:36.961949 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d92c2f-9379-43dd-8cff-19166c14b015 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:36.962367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d92c2f-9379-43dd-8cff-19166c14b015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:36.981334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d92c2f-9379-43dd-8cff-19166c14b015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:36.981859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d92c2f-9379-43dd-8cff-19166c14b015 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c4d92c2f-9379-43dd-8cff-19166c14b015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:36.996921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:36.997894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:36.998810 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:36.998810 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:36.998810 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:36.999004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519597c-1477-4c77-a7f4-491c5eb3a83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:36.999004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519597c-1477-4c77-a7f4-491c5eb3a83c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:36.999441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519597c-1477-4c77-a7f4-491c5eb3a83c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:36.999537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519597c-1477-4c77-a7f4-491c5eb3a83c None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.000135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519597c-1477-4c77-a7f4-491c5eb3a83c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:37.000437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519597c-1477-4c77-a7f4-491c5eb3a83c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:37.001788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519597c-1477-4c77-a7f4-491c5eb3a83c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:37.002844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 317/648] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:37.003713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d92c2f-9379-43dd-8cff-19166c14b015 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:37.019389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 332/649] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:37.019389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519597c-1477-4c77-a7f4-491c5eb3a83c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:37.117446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:37.130600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519597c-1477-4c77-a7f4-491c5eb3a83c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:37.131134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d92c2f-9379-43dd-8cff-19166c14b015 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:37.172373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f261a8-e21c-47cb-b86b-0c3129b4c35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:37.173243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f261a8-e21c-47cb-b86b-0c3129b4c35a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.173852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f261a8-e21c-47cb-b86b-0c3129b4c35a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.174200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f261a8-e21c-47cb-b86b-0c3129b4c35a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.181429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d48517-9ce7-479f-a45c-5fe4c5ef62dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:37.182310 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48517-9ce7-479f-a45c-5fe4c5ef62dd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.182588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48517-9ce7-479f-a45c-5fe4c5ef62dd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.182900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48517-9ce7-479f-a45c-5fe4c5ef62dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.187957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f261a8-e21c-47cb-b86b-0c3129b4c35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:37.189463 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f261a8-e21c-47cb-b86b-0c3129b4c35a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:37.197012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d48517-9ce7-479f-a45c-5fe4c5ef62dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:37.199081 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d48517-9ce7-479f-a45c-5fe4c5ef62dd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-58f261a8-e21c-47cb-b86b-0c3129b4c35a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:37.213744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:37.215713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:37.217158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:37.217158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:37.217158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-75d48517-9ce7-479f-a45c-5fe4c5ef62dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:37.217409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:37.218400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:37.221022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:37.221022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:37.221022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:37.221500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 333/650] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:37.222786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 318/651] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:37.222985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f261a8-e21c-47cb-b86b-0c3129b4c35a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:37.223653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d48517-9ce7-479f-a45c-5fe4c5ef62dd None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:37.250677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:37.340132 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d48517-9ce7-479f-a45c-5fe4c5ef62dd None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:37.355701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f261a8-e21c-47cb-b86b-0c3129b4c35a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:37.377326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de9961-1c1f-4c72-b64e-0f799a72dfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:37.377963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de9961-1c1f-4c72-b64e-0f799a72dfed None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.378306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de9961-1c1f-4c72-b64e-0f799a72dfed None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.378513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de9961-1c1f-4c72-b64e-0f799a72dfed None admin] PATH_INFO: `/v3/users/glance` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.386274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de9961-1c1f-4c72-b64e-0f799a72dfed None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:37.387769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de9961-1c1f-4c72-b64e-0f799a72dfed None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:37.390919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e961d-a3be-402a-832d-8f9f5b86332f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:37.391552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e961d-a3be-402a-832d-8f9f5b86332f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.391831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e961d-a3be-402a-832d-8f9f5b86332f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.392104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e961d-a3be-402a-832d-8f9f5b86332f None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.392738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e961d-a3be-402a-832d-8f9f5b86332f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:37.393040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e961d-a3be-402a-832d-8f9f5b86332f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:37.394380 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e961d-a3be-402a-832d-8f9f5b86332f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-58de9961-1c1f-4c72-b64e-0f799a72dfed None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:37.398881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:37.400440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 319/652] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:37.401388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de9961-1c1f-4c72-b64e-0f799a72dfed None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:37.410620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 334/653] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:37.411450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:37.411625 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e961d-a3be-402a-832d-8f9f5b86332f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:37.489694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:37.508317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de9961-1c1f-4c72-b64e-0f799a72dfed None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:37.514275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e961d-a3be-402a-832d-8f9f5b86332f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:37.541260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e53795-737b-4c67-b356-c84f4a9bcf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:37.541858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e53795-737b-4c67-b356-c84f4a9bcf35 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.542683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e53795-737b-4c67-b356-c84f4a9bcf35 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.542990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e53795-737b-4c67-b356-c84f4a9bcf35 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.543624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e53795-737b-4c67-b356-c84f4a9bcf35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:37.544019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e53795-737b-4c67-b356-c84f4a9bcf35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:37.545284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e53795-737b-4c67-b356-c84f4a9bcf35 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:37.549008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7149dd-5529-432f-b19d-78825d563166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:37.549613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7149dd-5529-432f-b19d-78825d563166 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.549893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7149dd-5529-432f-b19d-78825d563166 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.550167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7149dd-5529-432f-b19d-78825d563166 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.550798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7149dd-5529-432f-b19d-78825d563166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:37.551146 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7149dd-5529-432f-b19d-78825d563166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:37.552490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7149dd-5529-432f-b19d-78825d563166 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:37.557128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 320/654] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:37.558554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d245298-4017-4c35-b9db-ab260fd750ba None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.558841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d245298-4017-4c35-b9db-ab260fd750ba None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.559465 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d245298-4017-4c35-b9db-ab260fd750ba None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.560548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 321/655] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:37.561055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d245298-4017-4c35-b9db-ab260fd750ba None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:37.572739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 335/656] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:37.572739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7149dd-5529-432f-b19d-78825d563166 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:37.626557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:37.662353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d245298-4017-4c35-b9db-ab260fd750ba None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:37.670738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7149dd-5529-432f-b19d-78825d563166 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:37.697088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad51ea-7d25-44ab-9f25-d0dd9e314325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:37.697691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad51ea-7d25-44ab-9f25-d0dd9e314325 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.697973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad51ea-7d25-44ab-9f25-d0dd9e314325 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.698248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad51ea-7d25-44ab-9f25-d0dd9e314325 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.699567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad51ea-7d25-44ab-9f25-d0dd9e314325 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:37.699872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad51ea-7d25-44ab-9f25-d0dd9e314325 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:37.701302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad51ea-7d25-44ab-9f25-d0dd9e314325 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:37.704384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b842569-6138-4311-84b1-c9023cfbcae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:37.705005 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b842569-6138-4311-84b1-c9023cfbcae6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.705280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b842569-6138-4311-84b1-c9023cfbcae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.705544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b842569-6138-4311-84b1-c9023cfbcae6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.708189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:37.713583 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b842569-6138-4311-84b1-c9023cfbcae6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:37.715487 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b842569-6138-4311-84b1-c9023cfbcae6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:37.718482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 322/657] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 22 16:53:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:37.719336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad51ea-7d25-44ab-9f25-d0dd9e314325 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-0b842569-6138-4311-84b1-c9023cfbcae6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:37.726578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:37.727908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 336/658] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:37.728835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4966b-9c66-444a-9e3e-2e06b273f5f1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.729520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4966b-9c66-444a-9e3e-2e06b273f5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.729520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4966b-9c66-444a-9e3e-2e06b273f5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.748117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6ce4966b-9c66-444a-9e3e-2e06b273f5f1 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:37.749313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6ce4966b-9c66-444a-9e3e-2e06b273f5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:37.823067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 337/659] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:37.826575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4966b-9c66-444a-9e3e-2e06b273f5f1 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:37.838024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad51ea-7d25-44ab-9f25-d0dd9e314325 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:37.866546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:37.879429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2c777-8049-4e1d-9ebd-32c44709df34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:37.880055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2c777-8049-4e1d-9ebd-32c44709df34 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.880328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2c777-8049-4e1d-9ebd-32c44709df34 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.880594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2c777-8049-4e1d-9ebd-32c44709df34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.888541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2c777-8049-4e1d-9ebd-32c44709df34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:37.890088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2c777-8049-4e1d-9ebd-32c44709df34 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-00e2c777-8049-4e1d-9ebd-32c44709df34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:37.901727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:37.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:37.904047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:37.904047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:37.904047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:37.904047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:37.904047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:37.904047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:37.904047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:37.904047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:37.907333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 323/660] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:37.907949 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2c777-8049-4e1d-9ebd-32c44709df34 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:37.935399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4966b-9c66-444a-9e3e-2e06b273f5f1 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:37.964149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:37.977020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d758df3-503f-4eb4-8021-eb0605721c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:37.977931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d758df3-503f-4eb4-8021-eb0605721c54 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:37.978376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d758df3-503f-4eb4-8021-eb0605721c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:37.978928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d758df3-503f-4eb4-8021-eb0605721c54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:37.986651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d758df3-503f-4eb4-8021-eb0605721c54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:37.988106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d758df3-503f-4eb4-8021-eb0605721c54 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-2d758df3-503f-4eb4-8021-eb0605721c54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:38.000155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:38.001053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:38.001938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:38.001938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:38.001938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:38.001938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:38.001938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:38.001938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:38.001938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:38.001938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:38.006028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 338/661] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:38.006880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d758df3-503f-4eb4-8021-eb0605721c54 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:38.017653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2c777-8049-4e1d-9ebd-32c44709df34 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:38.056992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e6607-528e-4767-8cd5-d002aca37a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:38.058034 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e6607-528e-4767-8cd5-d002aca37a7f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:38.058525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e6607-528e-4767-8cd5-d002aca37a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:38.059017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e6607-528e-4767-8cd5-d002aca37a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:38.116825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d758df3-503f-4eb4-8021-eb0605721c54 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:38.117078 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:38.138921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e6607-528e-4767-8cd5-d002aca37a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:38.140401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e6607-528e-4767-8cd5-d002aca37a7f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:38.154927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e78b2-8fa3-4489-bdb3-210df988c6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:38.155567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e78b2-8fa3-4489-bdb3-210df988c6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:38.155847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e78b2-8fa3-4489-bdb3-210df988c6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:38.156121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e78b2-8fa3-4489-bdb3-210df988c6b9 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:38.156847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e78b2-8fa3-4489-bdb3-210df988c6b9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:38.157153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e78b2-8fa3-4489-bdb3-210df988c6b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:38.158566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e78b2-8fa3-4489-bdb3-210df988c6b9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:38.175635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 339/662] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Sep 22 16:53:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:38.176452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e78b2-8fa3-4489-bdb3-210df988c6b9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:38.226970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:38.243922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 324/663] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:38.244770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e6607-528e-4767-8cd5-d002aca37a7f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:38.284327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e78b2-8fa3-4489-bdb3-210df988c6b9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:38.318496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82dcec-6584-4750-b4d3-67dbac7eb1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:38.319131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82dcec-6584-4750-b4d3-67dbac7eb1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:38.319409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82dcec-6584-4750-b4d3-67dbac7eb1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:38.319679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82dcec-6584-4750-b4d3-67dbac7eb1d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:38.320385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82dcec-6584-4750-b4d3-67dbac7eb1d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:38.320690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82dcec-6584-4750-b4d3-67dbac7eb1d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:38.322140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82dcec-6584-4750-b4d3-67dbac7eb1d9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:38.331280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:38.338285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 340/664] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:38.338285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82dcec-6584-4750-b4d3-67dbac7eb1d9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:38.348029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e6607-528e-4767-8cd5-d002aca37a7f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:38.382411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46acef-94c9-4e71-8973-146a3b5e80bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:38.383041 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46acef-94c9-4e71-8973-146a3b5e80bb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:38.383322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46acef-94c9-4e71-8973-146a3b5e80bb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:38.383591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46acef-94c9-4e71-8973-146a3b5e80bb None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:38.384273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46acef-94c9-4e71-8973-146a3b5e80bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:38.384575 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46acef-94c9-4e71-8973-146a3b5e80bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:38.386058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46acef-94c9-4e71-8973-146a3b5e80bb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:38.397111 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 325/665] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:38.397894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46acef-94c9-4e71-8973-146a3b5e80bb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:38.447174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82dcec-6584-4750-b4d3-67dbac7eb1d9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:38.449515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:38.482044 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796e71e-4883-4d90-9185-96194183df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:38.482667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796e71e-4883-4d90-9185-96194183df3c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:38.482962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796e71e-4883-4d90-9185-96194183df3c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:38.483241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796e71e-4883-4d90-9185-96194183df3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:38.493395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796e71e-4883-4d90-9185-96194183df3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-1796e71e-4883-4d90-9185-96194183df3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:38.498839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:38.499901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:38.500752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 341/666] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:38.502097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46acef-94c9-4e71-8973-146a3b5e80bb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:38.502515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796e71e-4883-4d90-9185-96194183df3c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:38.537581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560dba-7474-4d20-ae52-c88c25c0a37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:38.538168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560dba-7474-4d20-ae52-c88c25c0a37a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:38.538393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560dba-7474-4d20-ae52-c88c25c0a37a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:38.538668 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560dba-7474-4d20-ae52-c88c25c0a37a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:38.553236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:38.606869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796e71e-4883-4d90-9185-96194183df3c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:38.607478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea560dba-7474-4d20-ae52-c88c25c0a37a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:38.608931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea560dba-7474-4d20-ae52-c88c25c0a37a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:38.639966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b1cfc-c0b0-4b50-8ecf-1e7d2adf5250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:38.640594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b1cfc-c0b0-4b50-8ecf-1e7d2adf5250 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:38.640878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b1cfc-c0b0-4b50-8ecf-1e7d2adf5250 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:38.641159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b1cfc-c0b0-4b50-8ecf-1e7d2adf5250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:38.652016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b1cfc-c0b0-4b50-8ecf-1e7d2adf5250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:38.653460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b1cfc-c0b0-4b50-8ecf-1e7d2adf5250 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e20b1cfc-c0b0-4b50-8ecf-1e7d2adf5250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:38.667912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:38.668949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:38.669842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:38.669842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:38.669842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:38.669842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 342/667] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:38.670457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b1cfc-c0b0-4b50-8ecf-1e7d2adf5250 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:38.688367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:38.709971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 326/668] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:38.711245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560dba-7474-4d20-ae52-c88c25c0a37a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:38.757750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:38.773751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b1cfc-c0b0-4b50-8ecf-1e7d2adf5250 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:38.811798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c0323-f2c6-4655-9928-5ccca309d32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:38.812317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c0323-f2c6-4655-9928-5ccca309d32f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:38.812596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c0323-f2c6-4655-9928-5ccca309d32f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:38.812866 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c0323-f2c6-4655-9928-5ccca309d32f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:38.813580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c0323-f2c6-4655-9928-5ccca309d32f None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=f6c1ce47d8704111988a78e625fd968c) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:38.813884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c0323-f2c6-4655-9928-5ccca309d32f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:38.815375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c0323-f2c6-4655-9928-5ccca309d32f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:38.816251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560dba-7474-4d20-ae52-c88c25c0a37a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:38.821155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 343/669] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:38 2022] GET /identity/v3/role_assignments?user.id=f6c1ce47d8704111988a78e625fd968c&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:38.822022 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c0323-f2c6-4655-9928-5ccca309d32f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:38.855919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50415af1-4204-496d-b4d3-11486c5c9f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:38.856544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50415af1-4204-496d-b4d3-11486c5c9f40 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:38.856826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50415af1-4204-496d-b4d3-11486c5c9f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:38.857106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50415af1-4204-496d-b4d3-11486c5c9f40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:38.857844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50415af1-4204-496d-b4d3-11486c5c9f40 None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=ae1b8a51c9fa4f2594bdd79b58962b0a) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:38.858158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50415af1-4204-496d-b4d3-11486c5c9f40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:38.859698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50415af1-4204-496d-b4d3-11486c5c9f40 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:38.871132 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 327/670] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:38 2022] GET /identity/v3/role_assignments?user.id=ae1b8a51c9fa4f2594bdd79b58962b0a&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:38.873122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50415af1-4204-496d-b4d3-11486c5c9f40 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:38.934162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:38.961075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c0323-f2c6-4655-9928-5ccca309d32f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:39.012272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaa1a2-4bc5-4bed-abbb-d1907b5d6615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:39.012904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaa1a2-4bc5-4bed-abbb-d1907b5d6615 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:39.013689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaa1a2-4bc5-4bed-abbb-d1907b5d6615 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:39.013950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaa1a2-4bc5-4bed-abbb-d1907b5d6615 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:39.014644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aaa1a2-4bc5-4bed-abbb-d1907b5d6615 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:39.014970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aaa1a2-4bc5-4bed-abbb-d1907b5d6615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:39.016397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aaa1a2-4bc5-4bed-abbb-d1907b5d6615 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:39.030010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 344/671] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:39.031021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaa1a2-4bc5-4bed-abbb-d1907b5d6615 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:39.034440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50415af1-4204-496d-b4d3-11486c5c9f40 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:39.046635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:39.081964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e290563-b8e0-4e48-baa2-f64a0d4eb3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:39.083121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e290563-b8e0-4e48-baa2-f64a0d4eb3fe None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:39.083121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e290563-b8e0-4e48-baa2-f64a0d4eb3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:39.083121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e290563-b8e0-4e48-baa2-f64a0d4eb3fe None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:39.083771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e290563-b8e0-4e48-baa2-f64a0d4eb3fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:39.084069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e290563-b8e0-4e48-baa2-f64a0d4eb3fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:39.085425 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e290563-b8e0-4e48-baa2-f64a0d4eb3fe None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:39.100507 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 328/672] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:39.101550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e290563-b8e0-4e48-baa2-f64a0d4eb3fe None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:39.177929 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaa1a2-4bc5-4bed-abbb-d1907b5d6615 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:39.217323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025668b0-adeb-42e4-b0f8-c3303062604e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:39.217964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025668b0-adeb-42e4-b0f8-c3303062604e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:39.218241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025668b0-adeb-42e4-b0f8-c3303062604e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:39.218507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025668b0-adeb-42e4-b0f8-c3303062604e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:39.231957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025668b0-adeb-42e4-b0f8-c3303062604e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:39.234008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025668b0-adeb-42e4-b0f8-c3303062604e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:39.239826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e290563-b8e0-4e48-baa2-f64a0d4eb3fe None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:39.241959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-025668b0-adeb-42e4-b0f8-c3303062604e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:39.252894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:39.253986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:39.254886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:39.254886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:39.254886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:39.254886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 345/673] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:39.256348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025668b0-adeb-42e4-b0f8-c3303062604e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:39.281005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece78a9b-ace7-4651-9e0b-418ec06e1147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:39.281647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece78a9b-ace7-4651-9e0b-418ec06e1147 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:39.281933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece78a9b-ace7-4651-9e0b-418ec06e1147 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:39.282205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece78a9b-ace7-4651-9e0b-418ec06e1147 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:39.296927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece78a9b-ace7-4651-9e0b-418ec06e1147 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:39.298391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece78a9b-ace7-4651-9e0b-418ec06e1147 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-ece78a9b-ace7-4651-9e0b-418ec06e1147 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:39.317780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:39.318817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:39.319732 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:39.319732 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:39.319732 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:39.323305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 329/674] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:39.323937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece78a9b-ace7-4651-9e0b-418ec06e1147 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:39.349355 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:39.379404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025668b0-adeb-42e4-b0f8-c3303062604e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:39.416488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2c1ef-164d-46d8-be24-6683e628b688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:39.417793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2c1ef-164d-46d8-be24-6683e628b688 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:39.418081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2c1ef-164d-46d8-be24-6683e628b688 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:39.418352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2c1ef-164d-46d8-be24-6683e628b688 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:39.426903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2c1ef-164d-46d8-be24-6683e628b688 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:39.428784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2c1ef-164d-46d8-be24-6683e628b688 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:39.440740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece78a9b-ace7-4651-9e0b-418ec06e1147 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-afc2c1ef-164d-46d8-be24-6683e628b688 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:39.449410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:39.450554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:39.452874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:39.452874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:39.452874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:39.452874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:39.452874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:39.452874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:39.452874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:39.452874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:39.452874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 346/675] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:39.452874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2c1ef-164d-46d8-be24-6683e628b688 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:39.478364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b21907-7cc0-4059-8b31-af7b580a71b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:39.479006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b21907-7cc0-4059-8b31-af7b580a71b6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:39.479290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b21907-7cc0-4059-8b31-af7b580a71b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:39.480094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b21907-7cc0-4059-8b31-af7b580a71b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:39.480745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b21907-7cc0-4059-8b31-af7b580a71b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:39.481049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b21907-7cc0-4059-8b31-af7b580a71b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:39.482428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b21907-7cc0-4059-8b31-af7b580a71b6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:39.496088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 330/676] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:39.497012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b21907-7cc0-4059-8b31-af7b580a71b6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:39.527325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:39.578643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2c1ef-164d-46d8-be24-6683e628b688 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:39.592262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:39.614850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b21907-7cc0-4059-8b31-af7b580a71b6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:39.620803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855d278-fc51-41ad-9e08-767b8f3aa39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:39.621420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855d278-fc51-41ad-9e08-767b8f3aa39d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:39.621701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855d278-fc51-41ad-9e08-767b8f3aa39d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:39.621971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855d278-fc51-41ad-9e08-767b8f3aa39d None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:39.622606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855d278-fc51-41ad-9e08-767b8f3aa39d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:39.623102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855d278-fc51-41ad-9e08-767b8f3aa39d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:39.624562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855d278-fc51-41ad-9e08-767b8f3aa39d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:39.641474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 347/677] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:39.642813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855d278-fc51-41ad-9e08-767b8f3aa39d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:39.659913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeffc0a-143e-4013-88df-8eff309b455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:39.660536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeffc0a-143e-4013-88df-8eff309b455c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:39.660816 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeffc0a-143e-4013-88df-8eff309b455c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:39.661089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeffc0a-143e-4013-88df-8eff309b455c None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:39.662071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeffc0a-143e-4013-88df-8eff309b455c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:39.662071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeffc0a-143e-4013-88df-8eff309b455c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:39.663640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeffc0a-143e-4013-88df-8eff309b455c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:39.677689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 331/678] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:39.678400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeffc0a-143e-4013-88df-8eff309b455c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:39.777091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855d278-fc51-41ad-9e08-767b8f3aa39d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:39.801000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:39.807665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeffc0a-143e-4013-88df-8eff309b455c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:39.830257 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86aba4-2159-42b7-887b-b0dedb54c49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:39.830646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86aba4-2159-42b7-887b-b0dedb54c49f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:39.830931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86aba4-2159-42b7-887b-b0dedb54c49f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:39.831227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86aba4-2159-42b7-887b-b0dedb54c49f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:39.840053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86aba4-2159-42b7-887b-b0dedb54c49f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:39.842995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86aba4-2159-42b7-887b-b0dedb54c49f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:39.856183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f59a9-e291-449f-bac7-11a6c1bb989d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:39.856836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f59a9-e291-449f-bac7-11a6c1bb989d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:39.857119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f59a9-e291-449f-bac7-11a6c1bb989d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:39.857396 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f59a9-e291-449f-bac7-11a6c1bb989d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-ab86aba4-2159-42b7-887b-b0dedb54c49f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:39.858131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:39.859259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:39.860187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:39.860187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:39.860187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:39.860187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:39.860187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:39.860187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:39.860187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:39.860187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:39.862542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 348/679] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:39.863833 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86aba4-2159-42b7-887b-b0dedb54c49f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:39.872250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f59a9-e291-449f-bac7-11a6c1bb989d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:39.875501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f59a9-e291-449f-bac7-11a6c1bb989d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:39.878876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-440f59a9-e291-449f-bac7-11a6c1bb989d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:39.891415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:39.892435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:39.893460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:39.893460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:39.893460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:39.893460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:39.893460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:39.893460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:39.893460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:39.893460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:39.893460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:39.893460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:39.897373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 332/680] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:39.898249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f59a9-e291-449f-bac7-11a6c1bb989d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:39.994890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86aba4-2159-42b7-887b-b0dedb54c49f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:40.003451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f59a9-e291-449f-bac7-11a6c1bb989d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:40.036120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e302a2-f97a-4ce2-b15d-d6002b932b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:40.037052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e302a2-f97a-4ce2-b15d-d6002b932b0f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:40.037537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e302a2-f97a-4ce2-b15d-d6002b932b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:40.038009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e302a2-f97a-4ce2-b15d-d6002b932b0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:40.038554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d76ff-4754-4b78-9944-e7efa27b4c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:40.039214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e302a2-f97a-4ce2-b15d-d6002b932b0f None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=6103e9a612c84a9ea7d21cc5128fa40b) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:40.039324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d76ff-4754-4b78-9944-e7efa27b4c3c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:40.039540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d76ff-4754-4b78-9944-e7efa27b4c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:40.039818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d76ff-4754-4b78-9944-e7efa27b4c3c None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:40.040239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e302a2-f97a-4ce2-b15d-d6002b932b0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:40.040506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d76ff-4754-4b78-9944-e7efa27b4c3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:40.040813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d76ff-4754-4b78-9944-e7efa27b4c3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:40.042302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d76ff-4754-4b78-9944-e7efa27b4c3c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:40.042513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e302a2-f97a-4ce2-b15d-d6002b932b0f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:40.050571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 349/681] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:39 2022] GET /identity/v3/role_assignments?user.id=6103e9a612c84a9ea7d21cc5128fa40b&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:40.051670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e302a2-f97a-4ce2-b15d-d6002b932b0f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:40.062257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 333/682] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:40.063497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d76ff-4754-4b78-9944-e7efa27b4c3c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:40.102061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:40.130224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:40.194535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d76ff-4754-4b78-9944-e7efa27b4c3c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:40.198088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e302a2-f97a-4ce2-b15d-d6002b932b0f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:40.256814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb288547-1f23-43f2-a638-33ec70df8904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:40.257450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb288547-1f23-43f2-a638-33ec70df8904 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:40.257740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb288547-1f23-43f2-a638-33ec70df8904 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:40.258022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb288547-1f23-43f2-a638-33ec70df8904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:40.258776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb288547-1f23-43f2-a638-33ec70df8904 None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=8c2ed1eb664348e384824f39ca90c950) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:40.259119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb288547-1f23-43f2-a638-33ec70df8904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:40.260613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb288547-1f23-43f2-a638-33ec70df8904 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:40.265838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54896c9-c75c-48cf-9a95-fa98e7f3e08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:40.266478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54896c9-c75c-48cf-9a95-fa98e7f3e08a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:40.266758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54896c9-c75c-48cf-9a95-fa98e7f3e08a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:40.267056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54896c9-c75c-48cf-9a95-fa98e7f3e08a None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:40.267807 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54896c9-c75c-48cf-9a95-fa98e7f3e08a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:40.268114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54896c9-c75c-48cf-9a95-fa98e7f3e08a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:40.269600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54896c9-c75c-48cf-9a95-fa98e7f3e08a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:40.272360 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 334/683] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:40 2022] GET /identity/v3/role_assignments?user.id=8c2ed1eb664348e384824f39ca90c950&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:40.307098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 350/684] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 22 16:53:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:40.313003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb288547-1f23-43f2-a638-33ec70df8904 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:40.442026 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb288547-1f23-43f2-a638-33ec70df8904 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:40.468265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:40.480223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53bff2-a6b1-4cb5-a0ed-52da1debe3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:40.480911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53bff2-a6b1-4cb5-a0ed-52da1debe3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:40.481192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53bff2-a6b1-4cb5-a0ed-52da1debe3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:40.481466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53bff2-a6b1-4cb5-a0ed-52da1debe3f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:40.555723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53bff2-a6b1-4cb5-a0ed-52da1debe3f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:40.557264 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53bff2-a6b1-4cb5-a0ed-52da1debe3f9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:40.662274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 335/685] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:40.668019 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54896c9-c75c-48cf-9a95-fa98e7f3e08a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:40.794403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:40.801320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54896c9-c75c-48cf-9a95-fa98e7f3e08a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:40.868491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f9609-9af7-4aa4-8880-621c2313f555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:40.868491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f9609-9af7-4aa4-8880-621c2313f555 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:40.868491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f9609-9af7-4aa4-8880-621c2313f555 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:40.868491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f9609-9af7-4aa4-8880-621c2313f555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:40.882828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f9609-9af7-4aa4-8880-621c2313f555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:40.884308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f9609-9af7-4aa4-8880-621c2313f555 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e43f9609-9af7-4aa4-8880-621c2313f555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:40.910934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:40.911974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:40.912930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:40.912930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:40.912930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:40.912930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 351/686] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:40.913172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53bff2-a6b1-4cb5-a0ed-52da1debe3f9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:40.952110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:41.057722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53bff2-a6b1-4cb5-a0ed-52da1debe3f9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:41.105124 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2460b6-eb13-49f9-a80e-bf3a3b9b542e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:41.105774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2460b6-eb13-49f9-a80e-bf3a3b9b542e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:41.106054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2460b6-eb13-49f9-a80e-bf3a3b9b542e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:41.106322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2460b6-eb13-49f9-a80e-bf3a3b9b542e None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:41.106980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2460b6-eb13-49f9-a80e-bf3a3b9b542e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:41.107288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2460b6-eb13-49f9-a80e-bf3a3b9b542e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:41.108662 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2460b6-eb13-49f9-a80e-bf3a3b9b542e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:41.124406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 336/687] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:41.130076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f9609-9af7-4aa4-8880-621c2313f555 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:41.282480 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f9609-9af7-4aa4-8880-621c2313f555 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:41.326523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c1a2-b151-4995-be34-2d59c2341655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:41.327570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c1a2-b151-4995-be34-2d59c2341655 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:41.328044 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c1a2-b151-4995-be34-2d59c2341655 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:41.328506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c1a2-b151-4995-be34-2d59c2341655 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:41.336074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629c1a2-b151-4995-be34-2d59c2341655 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:41.337919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629c1a2-b151-4995-be34-2d59c2341655 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b629c1a2-b151-4995-be34-2d59c2341655 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:41.359210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:41.360158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:41.361069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:41.361069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:41.361069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:41.361069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:41.361069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:41.361069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:41.361069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:41.361069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:41.363863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 352/688] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:41.372069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2460b6-eb13-49f9-a80e-bf3a3b9b542e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:41.500128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:41.524577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2460b6-eb13-49f9-a80e-bf3a3b9b542e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:41.576399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8f63f-dbc4-4326-8e6e-4f0c12dbac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:41.577354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f63f-dbc4-4326-8e6e-4f0c12dbac51 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:41.577786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f63f-dbc4-4326-8e6e-4f0c12dbac51 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:41.578205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f63f-dbc4-4326-8e6e-4f0c12dbac51 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:41.579069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8f63f-dbc4-4326-8e6e-4f0c12dbac51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:41.579521 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8f63f-dbc4-4326-8e6e-4f0c12dbac51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:41.581122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8f63f-dbc4-4326-8e6e-4f0c12dbac51 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:41.588830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7a194-0101-43ce-bad0-8d7dfd4992e8 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:41.589122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7a194-0101-43ce-bad0-8d7dfd4992e8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:41.589399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7a194-0101-43ce-bad0-8d7dfd4992e8 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:41.590993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 353/689] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:41.596072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae5148-809b-4956-b1d3-33c71a98a2c8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:41.596355 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae5148-809b-4956-b1d3-33c71a98a2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:41.596629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae5148-809b-4956-b1d3-33c71a98a2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:41.602110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 337/690] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:41.608205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8f63f-dbc4-4326-8e6e-4f0c12dbac51 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:41.636929 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f1ae5148-809b-4956-b1d3-33c71a98a2c8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:41.638401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f1ae5148-809b-4956-b1d3-33c71a98a2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:41.647329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:41.724315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 354/691] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:41.724730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8f63f-dbc4-4326-8e6e-4f0c12dbac51 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:41.725609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fd9c8-316d-4f21-8cc7-c1ce5a83f775 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:41.725896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fd9c8-316d-4f21-8cc7-c1ce5a83f775 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:41.726173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fd9c8-316d-4f21-8cc7-c1ce5a83f775 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:41.727935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 355/692] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:41.736275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64281d3e-bf85-4b91-b4b2-f57f6027ac61 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:41.736559 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64281d3e-bf85-4b91-b4b2-f57f6027ac61 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:41.736832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64281d3e-bf85-4b91-b4b2-f57f6027ac61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:41.760679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-64281d3e-bf85-4b91-b4b2-f57f6027ac61 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:41.764104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-64281d3e-bf85-4b91-b4b2-f57f6027ac61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:41.769891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5a71e-d09d-498b-a5cc-2e1f37443ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:41.770511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5a71e-d09d-498b-a5cc-2e1f37443ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:41.770792 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5a71e-d09d-498b-a5cc-2e1f37443ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:41.771233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5a71e-d09d-498b-a5cc-2e1f37443ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:41.772586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5a71e-d09d-498b-a5cc-2e1f37443ec6 None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=b1a54beb68ba4610976524684bc26d5b) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:41.773002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5a71e-d09d-498b-a5cc-2e1f37443ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:41.774982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5a71e-d09d-498b-a5cc-2e1f37443ec6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:41.786293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 338/693] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:41 2022] GET /identity/v3/role_assignments?user.id=b1a54beb68ba4610976524684bc26d5b&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:41.879707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 356/694] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:42.028868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5a71e-d09d-498b-a5cc-2e1f37443ec6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:42.041429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:42.136385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5a71e-d09d-498b-a5cc-2e1f37443ec6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:42.176783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eff07-90c6-41f7-9f0b-e0aad0474887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:42.177657 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eff07-90c6-41f7-9f0b-e0aad0474887 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:42.178081 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eff07-90c6-41f7-9f0b-e0aad0474887 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:42.178493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eff07-90c6-41f7-9f0b-e0aad0474887 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:42.191411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2eff07-90c6-41f7-9f0b-e0aad0474887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3e2eff07-90c6-41f7-9f0b-e0aad0474887 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:42.196978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:42.198146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:42.199237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:42.202061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 339/695] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:42.208445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64281d3e-bf85-4b91-b4b2-f57f6027ac61 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:42.223381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eff07-90c6-41f7-9f0b-e0aad0474887 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:42.302646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:42.304011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:42.347884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64281d3e-bf85-4b91-b4b2-f57f6027ac61 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:42.352276 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eff07-90c6-41f7-9f0b-e0aad0474887 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:42.387999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9000e1a-b344-4752-b63c-6c52f74d0d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:42.388960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000e1a-b344-4752-b63c-6c52f74d0d62 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:42.389440 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000e1a-b344-4752-b63c-6c52f74d0d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:42.389909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000e1a-b344-4752-b63c-6c52f74d0d62 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:42.390779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9000e1a-b344-4752-b63c-6c52f74d0d62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:42.391314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9000e1a-b344-4752-b63c-6c52f74d0d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:42.392990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9000e1a-b344-4752-b63c-6c52f74d0d62 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:42.397028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195b219-aaa1-460e-8f1a-9bda9f3ad744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:42.397659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195b219-aaa1-460e-8f1a-9bda9f3ad744 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:42.397946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195b219-aaa1-460e-8f1a-9bda9f3ad744 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:42.398228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195b219-aaa1-460e-8f1a-9bda9f3ad744 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:42.398808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195b219-aaa1-460e-8f1a-9bda9f3ad744 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:42.400225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195b219-aaa1-460e-8f1a-9bda9f3ad744 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:42.405099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 357/696] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:42.416570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9000e1a-b344-4752-b63c-6c52f74d0d62 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3195b219-aaa1-460e-8f1a-9bda9f3ad744 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:42.418903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:42.419936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:42.421155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 340/697] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Sep 22 16:53:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:42.426028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195b219-aaa1-460e-8f1a-9bda9f3ad744 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:42.545002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195b219-aaa1-460e-8f1a-9bda9f3ad744 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:42.582270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:42.590496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c9cfa-5a44-4b13-9ea0-7557e77c0cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:42.591552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c9cfa-5a44-4b13-9ea0-7557e77c0cbe None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:42.592042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c9cfa-5a44-4b13-9ea0-7557e77c0cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:42.592523 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c9cfa-5a44-4b13-9ea0-7557e77c0cbe None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:42.593386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c9cfa-5a44-4b13-9ea0-7557e77c0cbe None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:42.593892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c9cfa-5a44-4b13-9ea0-7557e77c0cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:42.595553 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c9cfa-5a44-4b13-9ea0-7557e77c0cbe None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:42.601848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 341/698] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 22 16:53:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:42.608376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c9cfa-5a44-4b13-9ea0-7557e77c0cbe None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:42.690836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9000e1a-b344-4752-b63c-6c52f74d0d62 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:42.718329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c9cfa-5a44-4b13-9ea0-7557e77c0cbe None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:42.743331 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:42.755103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd019bc-22ca-4899-9584-465540d5ac76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:42.755738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd019bc-22ca-4899-9584-465540d5ac76 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:42.756015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd019bc-22ca-4899-9584-465540d5ac76 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:42.756282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd019bc-22ca-4899-9584-465540d5ac76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:42.761149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c8d03-5624-48e8-8c68-3e89f48e4fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:42.762408 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c8d03-5624-48e8-8c68-3e89f48e4fcb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:42.762911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c8d03-5624-48e8-8c68-3e89f48e4fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:42.763426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c8d03-5624-48e8-8c68-3e89f48e4fcb None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:42.764282 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c8d03-5624-48e8-8c68-3e89f48e4fcb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:42.764800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c8d03-5624-48e8-8c68-3e89f48e4fcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:42.766388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c8d03-5624-48e8-8c68-3e89f48e4fcb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:42.773267 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 342/699] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 22 16:53:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:42.787997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd019bc-22ca-4899-9584-465540d5ac76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:42.789548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd019bc-22ca-4899-9584-465540d5ac76 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-fbd019bc-22ca-4899-9584-465540d5ac76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:42.812920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:42.814094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:42.815190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:42.815190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:42.815190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:42.819087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 358/700] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:42.820962 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c8d03-5624-48e8-8c68-3e89f48e4fcb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:42.927365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:42.943423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a04ab7-20a3-4822-8529-515cf8e01a73 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:42.944472 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a04ab7-20a3-4822-8529-515cf8e01a73 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:42.944957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a04ab7-20a3-4822-8529-515cf8e01a73 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:42.946983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 359/701] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:42.953023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60daa50-d2df-4309-8a7d-f44133156c16 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:42.953982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60daa50-d2df-4309-8a7d-f44133156c16 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:42.954496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60daa50-d2df-4309-8a7d-f44133156c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:42.965086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c8d03-5624-48e8-8c68-3e89f48e4fcb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:42.976336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d60daa50-d2df-4309-8a7d-f44133156c16 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:42.977883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d60daa50-d2df-4309-8a7d-f44133156c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:43.004307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaabc3-ad41-4d08-bbb4-f85d2dedf2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:43.004307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaabc3-ad41-4d08-bbb4-f85d2dedf2af None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:43.004307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaabc3-ad41-4d08-bbb4-f85d2dedf2af None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:43.004307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaabc3-ad41-4d08-bbb4-f85d2dedf2af None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:43.004724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdaabc3-ad41-4d08-bbb4-f85d2dedf2af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:43.004724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdaabc3-ad41-4d08-bbb4-f85d2dedf2af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:43.004724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdaabc3-ad41-4d08-bbb4-f85d2dedf2af None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:43.018215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 343/702] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:43.024038 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaabc3-ad41-4d08-bbb4-f85d2dedf2af None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:43.111202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 360/703] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:43.138577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaabc3-ad41-4d08-bbb4-f85d2dedf2af None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:43.160002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:43.175547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b90546-682d-452c-b877-1465a4ad4d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:43.176189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b90546-682d-452c-b877-1465a4ad4d41 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:43.176471 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b90546-682d-452c-b877-1465a4ad4d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:43.176739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b90546-682d-452c-b877-1465a4ad4d41 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:43.184641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b90546-682d-452c-b877-1465a4ad4d41 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:43.186173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b90546-682d-452c-b877-1465a4ad4d41 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-48b90546-682d-452c-b877-1465a4ad4d41 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:43.199856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:43.200879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:43.201841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:43.201841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:43.201841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:43.201841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:43.201841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:43.201841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:43.201841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:43.201841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:43.201841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:43.201841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:43.201841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 344/704] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 22 16:53:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:43.206791 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60daa50-d2df-4309-8a7d-f44133156c16 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:43.210997 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf9b92-e399-45e8-a487-eea52bdd22a3 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:43.214911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf9b92-e399-45e8-a487-eea52bdd22a3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:43.214911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf9b92-e399-45e8-a487-eea52bdd22a3 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:43.214911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 345/705] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:53:43.219169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe2769-dc1b-47d5-b9fa-39a4534c900f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:43.219462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe2769-dc1b-47d5-b9fa-39a4534c900f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:43.219741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe2769-dc1b-47d5-b9fa-39a4534c900f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:43.252031 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-49fe2769-dc1b-47d5-b9fa-39a4534c900f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:43.254381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-49fe2769-dc1b-47d5-b9fa-39a4534c900f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:43.346758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 346/706] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:43.374749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60daa50-d2df-4309-8a7d-f44133156c16 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:43.389654 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:43.411867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2e169-d9f0-4488-9916-0e31a0714499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:43.412869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2e169-d9f0-4488-9916-0e31a0714499 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:43.413325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2e169-d9f0-4488-9916-0e31a0714499 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:43.413743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2e169-d9f0-4488-9916-0e31a0714499 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:43.414613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2e169-d9f0-4488-9916-0e31a0714499 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:43.415097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2e169-d9f0-4488-9916-0e31a0714499 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:43.416670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2e169-d9f0-4488-9916-0e31a0714499 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:43.435934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 361/707] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 22 16:53:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:43.441772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe2769-dc1b-47d5-b9fa-39a4534c900f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:43.517323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2e169-d9f0-4488-9916-0e31a0714499 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:43.597528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe2769-dc1b-47d5-b9fa-39a4534c900f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:43.615557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:43.645699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fb03f-b68f-4849-a715-11d0007ff0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:43.646317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb03f-b68f-4849-a715-11d0007ff0dd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:43.646600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb03f-b68f-4849-a715-11d0007ff0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:43.647008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb03f-b68f-4849-a715-11d0007ff0dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:43.683605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2e169-d9f0-4488-9916-0e31a0714499 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:43.750422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a9c9f-68ee-4327-981c-4db7c956d62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:43.751086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a9c9f-68ee-4327-981c-4db7c956d62f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:43.751412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a9c9f-68ee-4327-981c-4db7c956d62f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:43.751688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a9c9f-68ee-4327-981c-4db7c956d62f None admin] PATH_INFO: `/v3/services/network` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:43.752278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a9c9f-68ee-4327-981c-4db7c956d62f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:43.753649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a9c9f-68ee-4327-981c-4db7c956d62f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-476a9c9f-68ee-4327-981c-4db7c956d62f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:43.771903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:43.772873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:43.773692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 362/708] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:43 2022] GET /identity/v3/services/network => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:43.775011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a9c9f-68ee-4327-981c-4db7c956d62f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:43.780790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fb03f-b68f-4849-a715-11d0007ff0dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:43.782292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fb03f-b68f-4849-a715-11d0007ff0dd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:43.824716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:43.902411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a9c9f-68ee-4327-981c-4db7c956d62f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:43.913588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 347/709] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:43.914489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fb03f-b68f-4849-a715-11d0007ff0dd None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:43.942894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f606c4f-7ecc-43fa-9cc9-26acd407eb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:43.943901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606c4f-7ecc-43fa-9cc9-26acd407eb00 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:43.944872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606c4f-7ecc-43fa-9cc9-26acd407eb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:43.945345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606c4f-7ecc-43fa-9cc9-26acd407eb00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:43.947338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:43.960860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f606c4f-7ecc-43fa-9cc9-26acd407eb00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:43.962677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f606c4f-7ecc-43fa-9cc9-26acd407eb00 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-3f606c4f-7ecc-43fa-9cc9-26acd407eb00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:43.978655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:43.980230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:43.981131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:43.981131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:43.981131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:43.983326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 363/710] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:43.984374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f606c4f-7ecc-43fa-9cc9-26acd407eb00 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:44.051912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fb03f-b68f-4849-a715-11d0007ff0dd None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:44.081396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:44.088118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e8953-3de9-47a0-8d50-38565c5241c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:44.088865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8953-3de9-47a0-8d50-38565c5241c8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:44.089266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8953-3de9-47a0-8d50-38565c5241c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:44.089546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8953-3de9-47a0-8d50-38565c5241c8 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:44.090241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e8953-3de9-47a0-8d50-38565c5241c8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:44.090612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e8953-3de9-47a0-8d50-38565c5241c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:44.092251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e8953-3de9-47a0-8d50-38565c5241c8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:44.101307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 348/711] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:53:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:44.101307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e8953-3de9-47a0-8d50-38565c5241c8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:44.107583 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f606c4f-7ecc-43fa-9cc9-26acd407eb00 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:44.155664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f0d8b-463c-4a7f-a81c-d868ee3dfa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:44.157136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f0d8b-463c-4a7f-a81c-d868ee3dfa3d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:44.157571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f0d8b-463c-4a7f-a81c-d868ee3dfa3d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:44.158015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f0d8b-463c-4a7f-a81c-d868ee3dfa3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:44.172330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f0d8b-463c-4a7f-a81c-d868ee3dfa3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:44.174504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f0d8b-463c-4a7f-a81c-d868ee3dfa3d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-5c1f0d8b-463c-4a7f-a81c-d868ee3dfa3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:44.191967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:44.193169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:44.194299 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:44.194299 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:44.194299 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:44.195286 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:44.196928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 364/712] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:44.198022 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f0d8b-463c-4a7f-a81c-d868ee3dfa3d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:44.228124 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e8953-3de9-47a0-8d50-38565c5241c8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:44.273195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e2759-90a4-4ea2-b533-c72fcd31f3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:44.274486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e2759-90a4-4ea2-b533-c72fcd31f3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:44.275191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e2759-90a4-4ea2-b533-c72fcd31f3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:44.275611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e2759-90a4-4ea2-b533-c72fcd31f3f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:44.276434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e2759-90a4-4ea2-b533-c72fcd31f3f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:44.276907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e2759-90a4-4ea2-b533-c72fcd31f3f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:44.278675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e2759-90a4-4ea2-b533-c72fcd31f3f6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:44.294145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 349/713] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:44.295216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e2759-90a4-4ea2-b533-c72fcd31f3f6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:44.325544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f0d8b-463c-4a7f-a81c-d868ee3dfa3d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:44.369860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fd89d-c629-4c0b-9cd9-af586a789bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:44.371557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fd89d-c629-4c0b-9cd9-af586a789bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:44.372012 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fd89d-c629-4c0b-9cd9-af586a789bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:44.372466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fd89d-c629-4c0b-9cd9-af586a789bc0 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:44.373336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fd89d-c629-4c0b-9cd9-af586a789bc0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:44.373801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fd89d-c629-4c0b-9cd9-af586a789bc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:44.375325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fd89d-c629-4c0b-9cd9-af586a789bc0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:44.391431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 365/714] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:53:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:44.392705 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fd89d-c629-4c0b-9cd9-af586a789bc0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:44.396498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:44.446975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e2759-90a4-4ea2-b533-c72fcd31f3f6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:44.478289 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1835fb-3173-4dcd-bcc0-2b5674bb80bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:44.478925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1835fb-3173-4dcd-bcc0-2b5674bb80bc None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:44.479233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1835fb-3173-4dcd-bcc0-2b5674bb80bc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:44.479574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1835fb-3173-4dcd-bcc0-2b5674bb80bc None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:44.480253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1835fb-3173-4dcd-bcc0-2b5674bb80bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:44.480556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1835fb-3173-4dcd-bcc0-2b5674bb80bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:44.482145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1835fb-3173-4dcd-bcc0-2b5674bb80bc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:44.484256 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:44.499724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 350/715] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:44.505767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1835fb-3173-4dcd-bcc0-2b5674bb80bc None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:44.625025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1835fb-3173-4dcd-bcc0-2b5674bb80bc None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:44.636266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fd89d-c629-4c0b-9cd9-af586a789bc0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:44.661891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303f1be-e15d-49a1-a7e5-1a7276d72551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:44.662653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303f1be-e15d-49a1-a7e5-1a7276d72551 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:44.662965 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303f1be-e15d-49a1-a7e5-1a7276d72551 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:44.663238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303f1be-e15d-49a1-a7e5-1a7276d72551 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:44.672738 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303f1be-e15d-49a1-a7e5-1a7276d72551 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:44.674624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303f1be-e15d-49a1-a7e5-1a7276d72551 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:44.680395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b0bdf-adc1-46c3-877e-1f03bd7a6a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:44.681018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b0bdf-adc1-46c3-877e-1f03bd7a6a8f None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:44.681309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b0bdf-adc1-46c3-877e-1f03bd7a6a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:44.681565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b0bdf-adc1-46c3-877e-1f03bd7a6a8f None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:44.682217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b0bdf-adc1-46c3-877e-1f03bd7a6a8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:44.684295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b0bdf-adc1-46c3-877e-1f03bd7a6a8f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-d303f1be-e15d-49a1-a7e5-1a7276d72551 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:44.687432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:44.688351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:44.689257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:44.689257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:44.689257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:44.689257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:44.689257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:44.689257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:44.689257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:44.689257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:44.689257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 351/716] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:44.690243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f977764-fd62-4e0a-abfd-37d713025e29 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:44.690243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f977764-fd62-4e0a-abfd-37d713025e29 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:44.692409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f977764-fd62-4e0a-abfd-37d713025e29 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:44.696730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 352/717] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:44.697359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f977764-fd62-4e0a-abfd-37d713025e29 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:44.710682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 366/718] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:53:44 2022] POST /identity/v3/users => generated 254 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:53:44.712181 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0018303-e7b2-4d96-9c6e-8e8655910c43 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:44.712475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0018303-e7b2-4d96-9c6e-8e8655910c43 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:44.712778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0018303-e7b2-4d96-9c6e-8e8655910c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:44.735047 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a0018303-e7b2-4d96-9c6e-8e8655910c43 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:44.736480 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a0018303-e7b2-4d96-9c6e-8e8655910c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:44.819336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:44.880494 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:44.907361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f977764-fd62-4e0a-abfd-37d713025e29 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:44.923890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 367/719] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:44.958375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa4463-2d89-4045-99eb-4c8c14e35a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:44.959074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa4463-2d89-4045-99eb-4c8c14e35a35 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:44.959356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa4463-2d89-4045-99eb-4c8c14e35a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:44.959625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa4463-2d89-4045-99eb-4c8c14e35a35 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:44.960313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa4463-2d89-4045-99eb-4c8c14e35a35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:44.960617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa4463-2d89-4045-99eb-4c8c14e35a35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:44.962050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa4463-2d89-4045-99eb-4c8c14e35a35 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:44.976155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 353/720] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:44.982163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0018303-e7b2-4d96-9c6e-8e8655910c43 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:45.091293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0018303-e7b2-4d96-9c6e-8e8655910c43 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:45.130590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f120ce4-ca4e-434b-a8be-8d285a596459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:45.131194 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f120ce4-ca4e-434b-a8be-8d285a596459 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:45.131518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f120ce4-ca4e-434b-a8be-8d285a596459 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:45.131800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f120ce4-ca4e-434b-a8be-8d285a596459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:45.132518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f120ce4-ca4e-434b-a8be-8d285a596459 None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=f6c1ce47d8704111988a78e625fd968c) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:45.132822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f120ce4-ca4e-434b-a8be-8d285a596459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:45.134243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f120ce4-ca4e-434b-a8be-8d285a596459 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:45.140541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 368/721] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:44 2022] GET /identity/v3/role_assignments?user.id=f6c1ce47d8704111988a78e625fd968c&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:45.256929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa4463-2d89-4045-99eb-4c8c14e35a35 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:45.363601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8d3a6-e719-4319-85ba-01015014ea7b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:45.364017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8d3a6-e719-4319-85ba-01015014ea7b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:45.364262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8d3a6-e719-4319-85ba-01015014ea7b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:45.365981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 369/722] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:53:45.374072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757213c-97b8-42c6-80d4-535d874be565 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:45.374357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757213c-97b8-42c6-80d4-535d874be565 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:45.374633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757213c-97b8-42c6-80d4-535d874be565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:45.401227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0757213c-97b8-42c6-80d4-535d874be565 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:45.402482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0757213c-97b8-42c6-80d4-535d874be565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:45.414016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa4463-2d89-4045-99eb-4c8c14e35a35 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:45.441989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:45.483334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:45.483538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7b976-4a73-4377-b2de-c5050d238964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:45.484163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7b976-4a73-4377-b2de-c5050d238964 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:45.484438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7b976-4a73-4377-b2de-c5050d238964 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:45.484705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7b976-4a73-4377-b2de-c5050d238964 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:45.507419 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba7b976-4a73-4377-b2de-c5050d238964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-0ba7b976-4a73-4377-b2de-c5050d238964 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:45.513393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:45.514299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:45.515187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:45.516281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 354/723] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:45.521476 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7b976-4a73-4377-b2de-c5050d238964 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:45.531789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 370/724] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:45.677722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7b976-4a73-4377-b2de-c5050d238964 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:45.736864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde384d-aa32-4f36-8675-64664f6a705c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:45.737488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde384d-aa32-4f36-8675-64664f6a705c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:45.737772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde384d-aa32-4f36-8675-64664f6a705c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:45.738043 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde384d-aa32-4f36-8675-64664f6a705c None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:45.738711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde384d-aa32-4f36-8675-64664f6a705c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:45.739036 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde384d-aa32-4f36-8675-64664f6a705c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:45.740413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde384d-aa32-4f36-8675-64664f6a705c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:45.761841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 355/725] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:45.770980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757213c-97b8-42c6-80d4-535d874be565 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:45.849880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde384d-aa32-4f36-8675-64664f6a705c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:45.874780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757213c-97b8-42c6-80d4-535d874be565 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:45.888905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:45.910685 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a88a7d-05a2-4618-bfbe-2e73eb358c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:45.911443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a88a7d-05a2-4618-bfbe-2e73eb358c64 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:45.911717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a88a7d-05a2-4618-bfbe-2e73eb358c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:45.911996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a88a7d-05a2-4618-bfbe-2e73eb358c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:45.922712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a88a7d-05a2-4618-bfbe-2e73eb358c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:45.924455 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a88a7d-05a2-4618-bfbe-2e73eb358c64 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-80a88a7d-05a2-4618-bfbe-2e73eb358c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:45.947367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:45.949014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:45.949930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:45.949930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:45.949930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:45.954091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 371/726] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:45.959888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a88a7d-05a2-4618-bfbe-2e73eb358c64 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:45.996650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde384d-aa32-4f36-8675-64664f6a705c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:46.024835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:46.064177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f17e8-43fb-421d-947d-17403f2a166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:46.065328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f17e8-43fb-421d-947d-17403f2a166b None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:46.065800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f17e8-43fb-421d-947d-17403f2a166b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:46.066269 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f17e8-43fb-421d-947d-17403f2a166b None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:46.067232 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f17e8-43fb-421d-947d-17403f2a166b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:46.068849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f17e8-43fb-421d-947d-17403f2a166b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:46.088969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 356/727] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:53:45 2022] POST /identity/v3/services => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 16:53:46.155185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a88a7d-05a2-4618-bfbe-2e73eb358c64 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:46.232584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cfa37-4533-499a-b8ea-9d2c403c1e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:46.233128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cfa37-4533-499a-b8ea-9d2c403c1e51 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:46.233408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cfa37-4533-499a-b8ea-9d2c403c1e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:46.233683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cfa37-4533-499a-b8ea-9d2c403c1e51 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:46.234334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cfa37-4533-499a-b8ea-9d2c403c1e51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:46.234636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cfa37-4533-499a-b8ea-9d2c403c1e51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:46.236075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cfa37-4533-499a-b8ea-9d2c403c1e51 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:46.255045 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 372/728] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:46.260255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f17e8-43fb-421d-947d-17403f2a166b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:46.406083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:46.415655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f17e8-43fb-421d-947d-17403f2a166b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:46.452925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f566b28-279e-4d7f-9ffd-5704b9416fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:46.453919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f566b28-279e-4d7f-9ffd-5704b9416fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:46.454409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f566b28-279e-4d7f-9ffd-5704b9416fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:46.454871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f566b28-279e-4d7f-9ffd-5704b9416fd5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:46.464385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f566b28-279e-4d7f-9ffd-5704b9416fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:46.466076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f566b28-279e-4d7f-9ffd-5704b9416fd5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-9f566b28-279e-4d7f-9ffd-5704b9416fd5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:46.478198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:46.479889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:46.480989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:46.480989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:46.480989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:46.480989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:46.480989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:46.480989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:46.480989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:46.480989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:46.480989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:46.480989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:46.483112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 357/729] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:46.492321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cfa37-4533-499a-b8ea-9d2c403c1e51 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:46.629868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:46.640738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cfa37-4533-499a-b8ea-9d2c403c1e51 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:46.689009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6651cb-6cfd-48fc-a19b-bff88585f27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:46.689619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6651cb-6cfd-48fc-a19b-bff88585f27d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:46.689901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6651cb-6cfd-48fc-a19b-bff88585f27d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:46.690171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6651cb-6cfd-48fc-a19b-bff88585f27d None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:46.690892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6651cb-6cfd-48fc-a19b-bff88585f27d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:46.691218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6651cb-6cfd-48fc-a19b-bff88585f27d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:46.692650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6651cb-6cfd-48fc-a19b-bff88585f27d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:46.723689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 373/730] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 22 16:53:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:46.729423 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f566b28-279e-4d7f-9ffd-5704b9416fd5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:46.946705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f566b28-279e-4d7f-9ffd-5704b9416fd5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:47.000822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b107d-3270-4b94-8502-24317ff30da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:47.001444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b107d-3270-4b94-8502-24317ff30da1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:47.001728 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b107d-3270-4b94-8502-24317ff30da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:47.001997 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b107d-3270-4b94-8502-24317ff30da1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:47.040150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:47.098539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b107d-3270-4b94-8502-24317ff30da1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:47.100019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b107d-3270-4b94-8502-24317ff30da1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:47.190159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52c47d-add5-49cf-b21c-9501c3fcd0c5 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:47.190673 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52c47d-add5-49cf-b21c-9501c3fcd0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:47.190673 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52c47d-add5-49cf-b21c-9501c3fcd0c5 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:47.193594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 374/731] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:47.197917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58755f-0c94-44f5-8d94-8c5e9814420d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:47.198736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58755f-0c94-44f5-8d94-8c5e9814420d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:47.199052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58755f-0c94-44f5-8d94-8c5e9814420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:47.223208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5b58755f-0c94-44f5-8d94-8c5e9814420d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:47.224584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5b58755f-0c94-44f5-8d94-8c5e9814420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:47.254914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 358/732] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:46 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 16:53:47.260386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b107d-3270-4b94-8502-24317ff30da1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:47.335139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 375/733] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 16:53:47.351829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:47.406647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b107d-3270-4b94-8502-24317ff30da1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:47.441815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddce962-67da-4a74-895c-9b3c022c5505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:47.442453 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddce962-67da-4a74-895c-9b3c022c5505 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:47.442731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddce962-67da-4a74-895c-9b3c022c5505 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:47.443097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddce962-67da-4a74-895c-9b3c022c5505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:47.453989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddce962-67da-4a74-895c-9b3c022c5505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:47.455557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddce962-67da-4a74-895c-9b3c022c5505 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-1ddce962-67da-4a74-895c-9b3c022c5505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:47.476422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:47.477418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:47.478302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:47.478302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:47.478302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:47.479948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 359/734] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:47.484691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58755f-0c94-44f5-8d94-8c5e9814420d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:47.573109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:47.601137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58755f-0c94-44f5-8d94-8c5e9814420d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:47.642559 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2392c-f006-415b-bd57-9a639f68b7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:47.643206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2392c-f006-415b-bd57-9a639f68b7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:47.643480 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2392c-f006-415b-bd57-9a639f68b7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:47.643747 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2392c-f006-415b-bd57-9a639f68b7d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:47.644389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2392c-f006-415b-bd57-9a639f68b7d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:47.644686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2392c-f006-415b-bd57-9a639f68b7d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:47.646037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2392c-f006-415b-bd57-9a639f68b7d2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:47.658320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 376/735] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:47.663698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddce962-67da-4a74-895c-9b3c022c5505 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:47.734184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81f035-2568-438c-8fce-22ea215a5ea4 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:47.734513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81f035-2568-438c-8fce-22ea215a5ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:47.734737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81f035-2568-438c-8fce-22ea215a5ea4 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:47.737153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 377/736] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:53:47.742564 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f0100-8acf-4954-9317-061e53fccf72 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:47.742849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f0100-8acf-4954-9317-061e53fccf72 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:47.743160 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f0100-8acf-4954-9317-061e53fccf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:47.764366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d70f0100-8acf-4954-9317-061e53fccf72 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:47.765779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d70f0100-8acf-4954-9317-061e53fccf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:47.814589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddce962-67da-4a74-895c-9b3c022c5505 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:47.827948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:47.863319 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e036a-c245-45c3-abc6-ef87776670b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:47.864364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e036a-c245-45c3-abc6-ef87776670b9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:47.864885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e036a-c245-45c3-abc6-ef87776670b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:47.865284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e036a-c245-45c3-abc6-ef87776670b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:47.872407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 378/737] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 16:53:47.873230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f0100-8acf-4954-9317-061e53fccf72 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:47.880899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e036a-c245-45c3-abc6-ef87776670b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:47.882367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e036a-c245-45c3-abc6-ef87776670b9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-898e036a-c245-45c3-abc6-ef87776670b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:47.902049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:47.903033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:47.903922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:47.903922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:47.903922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:47.903922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:47.903922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:47.903922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:47.903922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:47.903922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:47.905857 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 360/738] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:47.915998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e036a-c245-45c3-abc6-ef87776670b9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:47.980583 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:48.010132 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f0100-8acf-4954-9317-061e53fccf72 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:48.053689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ded9e1-b0b4-4d3b-a737-03178c98dc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:48.054809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ded9e1-b0b4-4d3b-a737-03178c98dc3c None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.055127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ded9e1-b0b4-4d3b-a737-03178c98dc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.055405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ded9e1-b0b4-4d3b-a737-03178c98dc3c None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:48.056072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ded9e1-b0b4-4d3b-a737-03178c98dc3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:48.057952 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ded9e1-b0b4-4d3b-a737-03178c98dc3c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:48.062188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e036a-c245-45c3-abc6-ef87776670b9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:48.083149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 379/739] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:53:47 2022] POST /identity/v3/services => generated 240 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:53:48.084646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f1fd3-cc3a-4b5b-82cf-f9d794e34e16 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.084932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f1fd3-cc3a-4b5b-82cf-f9d794e34e16 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.085268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f1fd3-cc3a-4b5b-82cf-f9d794e34e16 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:48.090984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 380/740] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:48.094058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32780014-58e1-42aa-a6d8-1a566ee795c5 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.094412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32780014-58e1-42aa-a6d8-1a566ee795c5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.094692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32780014-58e1-42aa-a6d8-1a566ee795c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:48.124289 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5ea07-20cf-47e4-a7fe-fb5c947fa566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:48.125235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5ea07-20cf-47e4-a7fe-fb5c947fa566 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.125710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5ea07-20cf-47e4-a7fe-fb5c947fa566 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.126163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5ea07-20cf-47e4-a7fe-fb5c947fa566 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:48.127092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5ea07-20cf-47e4-a7fe-fb5c947fa566 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:48.127614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5ea07-20cf-47e4-a7fe-fb5c947fa566 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:48.129681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5ea07-20cf-47e4-a7fe-fb5c947fa566 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:48.136161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-32780014-58e1-42aa-a6d8-1a566ee795c5 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:48.144691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-32780014-58e1-42aa-a6d8-1a566ee795c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:48.160757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 361/741] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:48.167239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5ea07-20cf-47e4-a7fe-fb5c947fa566 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:48.220221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:48.263155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 381/742] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:53:48.263155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32780014-58e1-42aa-a6d8-1a566ee795c5 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:48.326701 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5ea07-20cf-47e4-a7fe-fb5c947fa566 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:48.345363 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:48.379847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf528704-2ace-4678-b174-623cda7d4091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:48.380832 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf528704-2ace-4678-b174-623cda7d4091 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.381326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf528704-2ace-4678-b174-623cda7d4091 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.381862 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf528704-2ace-4678-b174-623cda7d4091 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:48.383134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf528704-2ace-4678-b174-623cda7d4091 None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=b1a54beb68ba4610976524684bc26d5b) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:48.383681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf528704-2ace-4678-b174-623cda7d4091 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:48.388175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf528704-2ace-4678-b174-623cda7d4091 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:48.398294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 362/743] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:48 2022] GET /identity/v3/role_assignments?user.id=b1a54beb68ba4610976524684bc26d5b&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:48.439297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32780014-58e1-42aa-a6d8-1a566ee795c5 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:48.480957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f5c87-48a0-4c71-81b2-9fc48793da35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:48.481618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5c87-48a0-4c71-81b2-9fc48793da35 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.481899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5c87-48a0-4c71-81b2-9fc48793da35 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.482170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5c87-48a0-4c71-81b2-9fc48793da35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:48.494605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f5c87-48a0-4c71-81b2-9fc48793da35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-853f5c87-48a0-4c71-81b2-9fc48793da35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:48.499858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:48.500832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:48.501665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 382/744] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:48.508803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf528704-2ace-4678-b174-623cda7d4091 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:48.589753 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f5c87-48a0-4c71-81b2-9fc48793da35 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:48.611914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:48.631725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf528704-2ace-4678-b174-623cda7d4091 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:48.675186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4f9f3-5698-4d69-b6de-1e251134c1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:48.675839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4f9f3-5698-4d69-b6de-1e251134c1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.676120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4f9f3-5698-4d69-b6de-1e251134c1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.676395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4f9f3-5698-4d69-b6de-1e251134c1a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:48.677258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4f9f3-5698-4d69-b6de-1e251134c1a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:48.677577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4f9f3-5698-4d69-b6de-1e251134c1a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:48.679001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4f9f3-5698-4d69-b6de-1e251134c1a8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:48.688659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 363/745] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:48.695994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4f9f3-5698-4d69-b6de-1e251134c1a8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:48.734548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f5c87-48a0-4c71-81b2-9fc48793da35 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:48.757066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:48.782165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1754ebf-a2dc-4226-bddf-bd9c148dfa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:48.782813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754ebf-a2dc-4226-bddf-bd9c148dfa7e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.783141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754ebf-a2dc-4226-bddf-bd9c148dfa7e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.783415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754ebf-a2dc-4226-bddf-bd9c148dfa7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:48.795998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1754ebf-a2dc-4226-bddf-bd9c148dfa7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e1754ebf-a2dc-4226-bddf-bd9c148dfa7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:48.802607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:48.803717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:48.804561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:48.809280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 383/746] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:48.810623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d134c-76b2-43ad-a3f6-870b8316a520 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.810910 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d134c-76b2-43ad-a3f6-870b8316a520 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.811910 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d134c-76b2-43ad-a3f6-870b8316a520 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:48.813652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 384/747] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:48.814395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d134c-76b2-43ad-a3f6-870b8316a520 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:48.846609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4f9f3-5698-4d69-b6de-1e251134c1a8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:48.890202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81720331-4e90-4273-869f-494007a0800f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:48.891443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81720331-4e90-4273-869f-494007a0800f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.892161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81720331-4e90-4273-869f-494007a0800f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.893054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81720331-4e90-4273-869f-494007a0800f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:48.909984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81720331-4e90-4273-869f-494007a0800f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:48.912354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81720331-4e90-4273-869f-494007a0800f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-81720331-4e90-4273-869f-494007a0800f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:48.934030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:48.935062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:48.936022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:48.936022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:48.936022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:48.941222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 364/748] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:48.941222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbe8e7-eeb3-4ae7-86af-898216c1e44a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.941222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbe8e7-eeb3-4ae7-86af-898216c1e44a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.941222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbe8e7-eeb3-4ae7-86af-898216c1e44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:48.948359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d134c-76b2-43ad-a3f6-870b8316a520 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:48.950223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:48.964749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-eefbe8e7-eeb3-4ae7-86af-898216c1e44a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:48.966006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-eefbe8e7-eeb3-4ae7-86af-898216c1e44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:48.996888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a9a7d-6fd6-4605-918c-412f7607356e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:48.998481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a9a7d-6fd6-4605-918c-412f7607356e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:48.999077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a9a7d-6fd6-4605-918c-412f7607356e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:48.999649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a9a7d-6fd6-4605-918c-412f7607356e None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:49.000640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a9a7d-6fd6-4605-918c-412f7607356e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:49.001186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a9a7d-6fd6-4605-918c-412f7607356e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:49.003969 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a9a7d-6fd6-4605-918c-412f7607356e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:49.016121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 385/749] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:49.017248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a9a7d-6fd6-4605-918c-412f7607356e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:49.047208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 365/750] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:53:49.048143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbe8e7-eeb3-4ae7-86af-898216c1e44a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:49.072081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:49.189317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a9a7d-6fd6-4605-918c-412f7607356e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:49.228670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbe8e7-eeb3-4ae7-86af-898216c1e44a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:49.253958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fef8f4-179d-4842-bbbc-53e3651fb574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:49.254577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef8f4-179d-4842-bbbc-53e3651fb574 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:49.254871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef8f4-179d-4842-bbbc-53e3651fb574 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:49.255153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef8f4-179d-4842-bbbc-53e3651fb574 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:49.256121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fef8f4-179d-4842-bbbc-53e3651fb574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:49.256629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fef8f4-179d-4842-bbbc-53e3651fb574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:49.259815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fef8f4-179d-4842-bbbc-53e3651fb574 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:49.284904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 386/751] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:49.287305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:49.292609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fef8f4-179d-4842-bbbc-53e3651fb574 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:49.299321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617f0e0-389f-4a6d-b0d0-123fca3f2475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:49.299983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617f0e0-389f-4a6d-b0d0-123fca3f2475 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:49.300263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617f0e0-389f-4a6d-b0d0-123fca3f2475 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:49.300539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617f0e0-389f-4a6d-b0d0-123fca3f2475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:49.320079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617f0e0-389f-4a6d-b0d0-123fca3f2475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:49.321681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617f0e0-389f-4a6d-b0d0-123fca3f2475 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-8617f0e0-389f-4a6d-b0d0-123fca3f2475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:49.339055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:49.340065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:49.340943 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:49.340943 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:49.340943 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:49.340943 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 366/752] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:49.345929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617f0e0-389f-4a6d-b0d0-123fca3f2475 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:49.435632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:49.438802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fef8f4-179d-4842-bbbc-53e3651fb574 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:49.478326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbd105-b8ed-4056-85b6-74f275264d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:49.480576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbd105-b8ed-4056-85b6-74f275264d58 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:49.481087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbd105-b8ed-4056-85b6-74f275264d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:49.481818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbd105-b8ed-4056-85b6-74f275264d58 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:49.491917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbd105-b8ed-4056-85b6-74f275264d58 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:49.493835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbd105-b8ed-4056-85b6-74f275264d58 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-11dbd105-b8ed-4056-85b6-74f275264d58 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:49.510638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:49.511712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:49.512651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:49.512651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:49.512651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:49.512651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:49.512651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:49.512651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:49.512651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:49.512651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:49.512651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:53:49.512651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:49.516728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 387/753] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 22 16:53:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:49.522189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbd105-b8ed-4056-85b6-74f275264d58 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:49.535052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617f0e0-389f-4a6d-b0d0-123fca3f2475 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:49.570528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f018d-c72a-42a8-92d5-361dbc460f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:49.571182 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f018d-c72a-42a8-92d5-361dbc460f0f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:49.571466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f018d-c72a-42a8-92d5-361dbc460f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:49.571741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f018d-c72a-42a8-92d5-361dbc460f0f None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:49.572385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f018d-c72a-42a8-92d5-361dbc460f0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:49.572691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f018d-c72a-42a8-92d5-361dbc460f0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:49.574139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f018d-c72a-42a8-92d5-361dbc460f0f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:49.591435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 367/754] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:49.592340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f018d-c72a-42a8-92d5-361dbc460f0f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:49.639386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:49.647625 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbd105-b8ed-4056-85b6-74f275264d58 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:49.696994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f21ded-9db4-4e19-8435-3172b785c96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:49.698547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f21ded-9db4-4e19-8435-3172b785c96a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:49.699294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f21ded-9db4-4e19-8435-3172b785c96a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:49.699740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f21ded-9db4-4e19-8435-3172b785c96a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:49.700817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f21ded-9db4-4e19-8435-3172b785c96a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:49.703095 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f21ded-9db4-4e19-8435-3172b785c96a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:49.710207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f018d-c72a-42a8-92d5-361dbc460f0f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-04f21ded-9db4-4e19-8435-3172b785c96a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:49.716370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:49.717356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 16:53:49.718173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:49.725351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 388/755] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Sep 22 16:53:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:49.725351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f21ded-9db4-4e19-8435-3172b785c96a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:49.757970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:49.771873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5923a-d971-4771-aea5-e769a9cd8e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:49.772728 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5923a-d971-4771-aea5-e769a9cd8e80 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:49.773185 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5923a-d971-4771-aea5-e769a9cd8e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:49.773620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5923a-d971-4771-aea5-e769a9cd8e80 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:49.774628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5923a-d971-4771-aea5-e769a9cd8e80 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:49.775146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5923a-d971-4771-aea5-e769a9cd8e80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:49.777418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5923a-d971-4771-aea5-e769a9cd8e80 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:49.799210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 368/756] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 22 16:53:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:49.800390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5923a-d971-4771-aea5-e769a9cd8e80 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:49.840808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f21ded-9db4-4e19-8435-3172b785c96a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:49.878589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098af0a1-51c9-4bb0-93cf-a45563b26cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:49.879273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098af0a1-51c9-4bb0-93cf-a45563b26cfc None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:49.879551 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098af0a1-51c9-4bb0-93cf-a45563b26cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:49.879823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098af0a1-51c9-4bb0-93cf-a45563b26cfc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:49.892035 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098af0a1-51c9-4bb0-93cf-a45563b26cfc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:49.893488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098af0a1-51c9-4bb0-93cf-a45563b26cfc None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-098af0a1-51c9-4bb0-93cf-a45563b26cfc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:49.906148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:49.907202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:49.908159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:49.908159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:49.908159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:49.908159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:49.908159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:49.908159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:49.908159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:49.908159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:49.908159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:49.908159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:49.912157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 389/757] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 22 16:53:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:49.912985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098af0a1-51c9-4bb0-93cf-a45563b26cfc None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:49.925298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5923a-d971-4771-aea5-e769a9cd8e80 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:49.927184 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:49.968860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369705ed-c54f-4ce3-b0dc-c14ee8c588f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:49.969473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369705ed-c54f-4ce3-b0dc-c14ee8c588f4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:49.969755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369705ed-c54f-4ce3-b0dc-c14ee8c588f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:49.970179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369705ed-c54f-4ce3-b0dc-c14ee8c588f4 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:49.970813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369705ed-c54f-4ce3-b0dc-c14ee8c588f4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:49.971429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369705ed-c54f-4ce3-b0dc-c14ee8c588f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:49.972700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369705ed-c54f-4ce3-b0dc-c14ee8c588f4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:49.977721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 369/758] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 22 16:53:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:49.978713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369705ed-c54f-4ce3-b0dc-c14ee8c588f4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:50.008975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:50.027085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098af0a1-51c9-4bb0-93cf-a45563b26cfc None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:50.072696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830eaae5-694c-4f51-8672-95a464591530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:50.073700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830eaae5-694c-4f51-8672-95a464591530 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.074123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830eaae5-694c-4f51-8672-95a464591530 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.074567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830eaae5-694c-4f51-8672-95a464591530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.103930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369705ed-c54f-4ce3-b0dc-c14ee8c588f4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:50.147594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8ab7c-6a75-481e-b41b-9286f7e83ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:50.148207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8ab7c-6a75-481e-b41b-9286f7e83ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.148487 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8ab7c-6a75-481e-b41b-9286f7e83ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.148760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8ab7c-6a75-481e-b41b-9286f7e83ae8 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.149763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8ab7c-6a75-481e-b41b-9286f7e83ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:50.150070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8ab7c-6a75-481e-b41b-9286f7e83ae8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:50.150274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830eaae5-694c-4f51-8672-95a464591530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:50.151557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8ab7c-6a75-481e-b41b-9286f7e83ae8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:50.152097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830eaae5-694c-4f51-8672-95a464591530 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:50.174107 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 370/759] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 22 16:53:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:50.175052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8ab7c-6a75-481e-b41b-9286f7e83ae8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:50.209054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:50.263309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:50.284835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 390/760] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:53:50.286144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830eaae5-694c-4f51-8672-95a464591530 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:50.292448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8ab7c-6a75-481e-b41b-9286f7e83ae8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:50.328497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4783067-396e-4b13-9c32-e4e4df97a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:50.329061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4783067-396e-4b13-9c32-e4e4df97a53e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.329433 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4783067-396e-4b13-9c32-e4e4df97a53e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.329886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4783067-396e-4b13-9c32-e4e4df97a53e None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.330536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4783067-396e-4b13-9c32-e4e4df97a53e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:50.330840 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4783067-396e-4b13-9c32-e4e4df97a53e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:50.332527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4783067-396e-4b13-9c32-e4e4df97a53e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:50.343548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 371/761] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 22 16:53:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:50.344814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4783067-396e-4b13-9c32-e4e4df97a53e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:50.408246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830eaae5-694c-4f51-8672-95a464591530 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:50.441877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:50.448530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ca543-1513-4ee1-9c83-e1bd3725be95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:50.449165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca543-1513-4ee1-9c83-e1bd3725be95 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.449446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca543-1513-4ee1-9c83-e1bd3725be95 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.449714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca543-1513-4ee1-9c83-e1bd3725be95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.457695 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4783067-396e-4b13-9c32-e4e4df97a53e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:50.462109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ca543-1513-4ee1-9c83-e1bd3725be95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:50.464144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ca543-1513-4ee1-9c83-e1bd3725be95 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-274ca543-1513-4ee1-9c83-e1bd3725be95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:50.482313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:50.483336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:50.484221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:50.484221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:50.484221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:50.484221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 391/762] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:50.484873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ca543-1513-4ee1-9c83-e1bd3725be95 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:50.499247 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139ae57-0fe7-4bc6-90ac-a46eb64d71ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:50.500335 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139ae57-0fe7-4bc6-90ac-a46eb64d71ac None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.500806 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139ae57-0fe7-4bc6-90ac-a46eb64d71ac None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.501267 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139ae57-0fe7-4bc6-90ac-a46eb64d71ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.512518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139ae57-0fe7-4bc6-90ac-a46eb64d71ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:50.514417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139ae57-0fe7-4bc6-90ac-a46eb64d71ac None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:50.527686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-7139ae57-0fe7-4bc6-90ac-a46eb64d71ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:50.532535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:50.533462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:50.534367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:50.534367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:50.534367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:50.534367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 372/763] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:50.535097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139ae57-0fe7-4bc6-90ac-a46eb64d71ac None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:50.615036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ca543-1513-4ee1-9c83-e1bd3725be95 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:50.650016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139ae57-0fe7-4bc6-90ac-a46eb64d71ac None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:50.658931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e141b-d149-4a77-b086-3280f0f9d309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:50.660351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e141b-d149-4a77-b086-3280f0f9d309 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.661292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e141b-d149-4a77-b086-3280f0f9d309 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.661766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e141b-d149-4a77-b086-3280f0f9d309 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.662902 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e141b-d149-4a77-b086-3280f0f9d309 None admin] RBAC: Adding query filter params (interface=public, service_id=019e6b69f48840f68f61ebf844d567d6, region_id=RegionOne) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:50.663437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e141b-d149-4a77-b086-3280f0f9d309 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:50.665172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e141b-d149-4a77-b086-3280f0f9d309 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:50.679941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 392/764] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 22 16:53:50 2022] GET /identity/v3/endpoints?service_id=019e6b69f48840f68f61ebf844d567d6&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:50.680737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e141b-d149-4a77-b086-3280f0f9d309 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:50.688542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815c666-e858-4bc6-937a-3cd4bbdd7c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:50.689180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815c666-e858-4bc6-937a-3cd4bbdd7c0b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.689460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815c666-e858-4bc6-937a-3cd4bbdd7c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.689733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815c666-e858-4bc6-937a-3cd4bbdd7c0b None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.690400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815c666-e858-4bc6-937a-3cd4bbdd7c0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:50.690706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815c666-e858-4bc6-937a-3cd4bbdd7c0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:50.692215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815c666-e858-4bc6-937a-3cd4bbdd7c0b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:50.702742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:50.707152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 373/765] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:50.708600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364bd46-d481-40e9-ae8a-7840a0de8cdb None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.710050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364bd46-d481-40e9-ae8a-7840a0de8cdb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.710325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364bd46-d481-40e9-ae8a-7840a0de8cdb None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.714993 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 374/766] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:50.714993 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364bd46-d481-40e9-ae8a-7840a0de8cdb None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:50.805896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:50.809264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e141b-d149-4a77-b086-3280f0f9d309 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:50.825775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364bd46-d481-40e9-ae8a-7840a0de8cdb None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:50.843716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d887029-c70a-40f1-bde5-990b1eb27023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:50.844282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d887029-c70a-40f1-bde5-990b1eb27023 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.844564 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d887029-c70a-40f1-bde5-990b1eb27023 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.844841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d887029-c70a-40f1-bde5-990b1eb27023 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.845478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d887029-c70a-40f1-bde5-990b1eb27023 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:50.845782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d887029-c70a-40f1-bde5-990b1eb27023 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:50.847177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d887029-c70a-40f1-bde5-990b1eb27023 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:50.859741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 393/767] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:50.860549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d887029-c70a-40f1-bde5-990b1eb27023 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:50.861957 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc8650-aa62-42df-80ad-3385d556fa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:50.862630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc8650-aa62-42df-80ad-3385d556fa1f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.862916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc8650-aa62-42df-80ad-3385d556fa1f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.863282 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc8650-aa62-42df-80ad-3385d556fa1f None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.863908 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc8650-aa62-42df-80ad-3385d556fa1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:50.865249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc8650-aa62-42df-80ad-3385d556fa1f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:50.870996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 375/768] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 22 16:53:50 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:50.872163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be9565-341a-4d22-aa13-bdda2531ef0d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.872492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be9565-341a-4d22-aa13-bdda2531ef0d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.872789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be9565-341a-4d22-aa13-bdda2531ef0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.896402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f5be9565-341a-4d22-aa13-bdda2531ef0d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:50.897686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f5be9565-341a-4d22-aa13-bdda2531ef0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:50.967366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:50.980243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 376/769] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:53:50.981350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554e090-8163-4a02-aebb-dfe4fb36c516 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:50.981577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d887029-c70a-40f1-bde5-990b1eb27023 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:50.981703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554e090-8163-4a02-aebb-dfe4fb36c516 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:50.981898 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554e090-8163-4a02-aebb-dfe4fb36c516 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:50.983162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 377/770] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:50.983877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554e090-8163-4a02-aebb-dfe4fb36c516 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:51.033852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82479079-932e-40a0-95b7-54e637d4630a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:51.034498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82479079-932e-40a0-95b7-54e637d4630a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:51.034787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82479079-932e-40a0-95b7-54e637d4630a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:51.035082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82479079-932e-40a0-95b7-54e637d4630a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:51.050037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82479079-932e-40a0-95b7-54e637d4630a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:51.051575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82479079-932e-40a0-95b7-54e637d4630a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-82479079-932e-40a0-95b7-54e637d4630a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:51.065769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:51.066704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:51.067631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:51.067631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:51.067631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:51.067631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:51.067631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:51.067631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:51.067631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:51.067631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:51.067631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 394/771] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:51.068415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1440a7-738e-4daa-b233-6b1c45121023 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:51.068668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1440a7-738e-4daa-b233-6b1c45121023 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:51.068939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1440a7-738e-4daa-b233-6b1c45121023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:51.087643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6e1440a7-738e-4daa-b233-6b1c45121023 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:51.090442 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6e1440a7-738e-4daa-b233-6b1c45121023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:51.110058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:51.137229 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554e090-8163-4a02-aebb-dfe4fb36c516 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:51.180532 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb5443-caf3-487e-bbac-877943fc72ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:51.181751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb5443-caf3-487e-bbac-877943fc72ac None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:51.182299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb5443-caf3-487e-bbac-877943fc72ac None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:51.182864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb5443-caf3-487e-bbac-877943fc72ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:51.185051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 395/772] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:53:51.185986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1440a7-738e-4daa-b233-6b1c45121023 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:51.192837 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb5443-caf3-487e-bbac-877943fc72ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:51.195266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb5443-caf3-487e-bbac-877943fc72ac None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-1dfb5443-caf3-487e-bbac-877943fc72ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:51.212172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:51.213125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:51.214582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:51.214582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:51.214582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:51.214582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:51.214582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:51.214582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:51.214582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:51.214582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:51.217367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 378/773] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:51.223048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb5443-caf3-487e-bbac-877943fc72ac None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:51.308496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:51.336071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1440a7-738e-4daa-b233-6b1c45121023 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:51.373735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb5443-caf3-487e-bbac-877943fc72ac None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:51.381315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116e13f-d0cf-40d0-9928-ae730908f2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:51.381986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116e13f-d0cf-40d0-9928-ae730908f2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:51.382331 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116e13f-d0cf-40d0-9928-ae730908f2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:51.382611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116e13f-d0cf-40d0-9928-ae730908f2d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:51.383333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116e13f-d0cf-40d0-9928-ae730908f2d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:51.383636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116e13f-d0cf-40d0-9928-ae730908f2d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:51.386209 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116e13f-d0cf-40d0-9928-ae730908f2d4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:51.404450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 396/774] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:51.404450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116e13f-d0cf-40d0-9928-ae730908f2d4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:51.420971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:51.424964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7e613-7d65-4dc7-a09d-830edbc9abbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:51.426923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e613-7d65-4dc7-a09d-830edbc9abbb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:51.427709 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e613-7d65-4dc7-a09d-830edbc9abbb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:51.428193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e613-7d65-4dc7-a09d-830edbc9abbb None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:51.429322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7e613-7d65-4dc7-a09d-830edbc9abbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:51.430185 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7e613-7d65-4dc7-a09d-830edbc9abbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:51.431846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7e613-7d65-4dc7-a09d-830edbc9abbb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:51.450407 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 379/775] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:51.451556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7e613-7d65-4dc7-a09d-830edbc9abbb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:51.522719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116e13f-d0cf-40d0-9928-ae730908f2d4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:51.560413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943df36-6f85-4299-b384-e3aa2abc80af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:51.561524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943df36-6f85-4299-b384-e3aa2abc80af None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:51.562049 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943df36-6f85-4299-b384-e3aa2abc80af None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:51.562495 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943df36-6f85-4299-b384-e3aa2abc80af None admin] PATH_INFO: `/v3/services/network` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:51.563252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943df36-6f85-4299-b384-e3aa2abc80af None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:51.564746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943df36-6f85-4299-b384-e3aa2abc80af None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:51.568545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7e613-7d65-4dc7-a09d-830edbc9abbb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-5943df36-6f85-4299-b384-e3aa2abc80af None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:51.574509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:51.575648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 16:53:51.576675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:51.579157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 397/776] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:51 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:51.580180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943df36-6f85-4299-b384-e3aa2abc80af None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:51.586825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:51.608518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d28b4c-711a-484c-915c-461c76fab988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:51.609277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d28b4c-711a-484c-915c-461c76fab988 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:51.609557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d28b4c-711a-484c-915c-461c76fab988 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:51.609827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d28b4c-711a-484c-915c-461c76fab988 None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/f6c1ce47d8704111988a78e625fd968c/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:51.611895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d28b4c-711a-484c-915c-461c76fab988 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=f6c1ce47d8704111988a78e625fd968c, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:51.613581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d28b4c-711a-484c-915c-461c76fab988 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:51.632285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 380/777] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:53:51 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/f6c1ce47d8704111988a78e625fd968c/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:53:51.639744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d28b4c-711a-484c-915c-461c76fab988 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:51.674350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:51.713757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943df36-6f85-4299-b384-e3aa2abc80af None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:51.756717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e30194-e2bc-4057-baeb-03aaf259c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:51.757339 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e30194-e2bc-4057-baeb-03aaf259c845 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:51.757725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e30194-e2bc-4057-baeb-03aaf259c845 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:51.757999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e30194-e2bc-4057-baeb-03aaf259c845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:51.761721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e30194-e2bc-4057-baeb-03aaf259c845 None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=abbbe2b3f23b4c1caf8e20825cc2cfee) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:51.762132 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e30194-e2bc-4057-baeb-03aaf259c845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:51.763642 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e30194-e2bc-4057-baeb-03aaf259c845 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:51.779452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 398/778] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:51 2022] GET /identity/v3/role_assignments?user.id=abbbe2b3f23b4c1caf8e20825cc2cfee&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:51.780490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e30194-e2bc-4057-baeb-03aaf259c845 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:51.784977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d28b4c-711a-484c-915c-461c76fab988 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:51.828032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece4774-5f42-4c2c-8715-d60fb298e7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:51.829076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece4774-5f42-4c2c-8715-d60fb298e7cf None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:51.829556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece4774-5f42-4c2c-8715-d60fb298e7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:51.830051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece4774-5f42-4c2c-8715-d60fb298e7cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:51.846569 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece4774-5f42-4c2c-8715-d60fb298e7cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-eece4774-5f42-4c2c-8715-d60fb298e7cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:51.852030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:51.853118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:51.853948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:51.862606 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 381/779] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:51.865038 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece4774-5f42-4c2c-8715-d60fb298e7cf None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:51.917555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:51.935044 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e30194-e2bc-4057-baeb-03aaf259c845 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:51.998768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f02686-ba49-4004-9e01-31628c2cd593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:51.999444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f02686-ba49-4004-9e01-31628c2cd593 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:51.999721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f02686-ba49-4004-9e01-31628c2cd593 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:51.999994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f02686-ba49-4004-9e01-31628c2cd593 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:52.000625 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f02686-ba49-4004-9e01-31628c2cd593 None admin] RBAC: Adding query filter params (name=network) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:52.000922 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f02686-ba49-4004-9e01-31628c2cd593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:52.002292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f02686-ba49-4004-9e01-31628c2cd593 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:52.018140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 399/780] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:53:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:52.021547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f02686-ba49-4004-9e01-31628c2cd593 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:52.024960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:52.030901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece4774-5f42-4c2c-8715-d60fb298e7cf None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:52.091467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e361d-ee08-491d-bfdb-6f12a8fc727f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:52.091959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e361d-ee08-491d-bfdb-6f12a8fc727f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:52.092238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e361d-ee08-491d-bfdb-6f12a8fc727f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:52.092555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e361d-ee08-491d-bfdb-6f12a8fc727f None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:52.093235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e361d-ee08-491d-bfdb-6f12a8fc727f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:52.093620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e361d-ee08-491d-bfdb-6f12a8fc727f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:52.095240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e361d-ee08-491d-bfdb-6f12a8fc727f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:52.106306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 382/781] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:52.112446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e361d-ee08-491d-bfdb-6f12a8fc727f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:52.196679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f02686-ba49-4004-9e01-31628c2cd593 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:52.234269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d36e9b-4336-4545-b028-7b5aa80788ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:52.234887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d36e9b-4336-4545-b028-7b5aa80788ff None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:52.235222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d36e9b-4336-4545-b028-7b5aa80788ff None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:52.235499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d36e9b-4336-4545-b028-7b5aa80788ff None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:52.236139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d36e9b-4336-4545-b028-7b5aa80788ff None admin] RBAC: Adding query filter params (type=network) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:52.236443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d36e9b-4336-4545-b028-7b5aa80788ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:52.237856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d36e9b-4336-4545-b028-7b5aa80788ff None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:52.243797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 400/782] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:53:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:52.252843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d36e9b-4336-4545-b028-7b5aa80788ff None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:52.332498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:52.348853 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:52.360298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e361d-ee08-491d-bfdb-6f12a8fc727f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:52.399591 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d36e9b-4336-4545-b028-7b5aa80788ff None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:52.437932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea3641-b243-4220-a7c6-55ec58bca308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:52.438545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3641-b243-4220-a7c6-55ec58bca308 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:52.438821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3641-b243-4220-a7c6-55ec58bca308 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:52.439140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3641-b243-4220-a7c6-55ec58bca308 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:52.440300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea3641-b243-4220-a7c6-55ec58bca308 None admin] RBAC: Adding query filter params (interface=public, service_id=790309d1eb0649e3b06c3f40f88f48df, region_id=RegionOne) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:52.440613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea3641-b243-4220-a7c6-55ec58bca308 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:52.442293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea3641-b243-4220-a7c6-55ec58bca308 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:52.446876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c655696-267c-4e3c-b6d5-7f23426f9044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:52.448322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655696-267c-4e3c-b6d5-7f23426f9044 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:52.448775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655696-267c-4e3c-b6d5-7f23426f9044 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:52.449192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655696-267c-4e3c-b6d5-7f23426f9044 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:52.454381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 401/783] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 22 16:53:52 2022] GET /identity/v3/endpoints?service_id=790309d1eb0649e3b06c3f40f88f48df&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:52.457080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea3641-b243-4220-a7c6-55ec58bca308 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:52.466335 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c655696-267c-4e3c-b6d5-7f23426f9044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:52.468052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c655696-267c-4e3c-b6d5-7f23426f9044 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-9c655696-267c-4e3c-b6d5-7f23426f9044 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:52.493351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:52.494358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:52.495839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:52.495839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:52.495839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:52.499611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 383/784] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:52.500497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c655696-267c-4e3c-b6d5-7f23426f9044 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:52.575781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea3641-b243-4220-a7c6-55ec58bca308 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:52.628807 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:52.640678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d962a0b-3845-400d-b6d9-c592122e29c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:52.642971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d962a0b-3845-400d-b6d9-c592122e29c9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:52.644767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d962a0b-3845-400d-b6d9-c592122e29c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:52.645276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d962a0b-3845-400d-b6d9-c592122e29c9 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:52.646087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d962a0b-3845-400d-b6d9-c592122e29c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:52.649853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d962a0b-3845-400d-b6d9-c592122e29c9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:52.669807 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 402/785] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 22 16:53:52 2022] GET /identity/v3/services => generated 772 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:52.709390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c655696-267c-4e3c-b6d5-7f23426f9044 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:52.761520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08c915-0925-41be-9887-5176d538c690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:52.762571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08c915-0925-41be-9887-5176d538c690 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:52.763081 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08c915-0925-41be-9887-5176d538c690 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:52.763637 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08c915-0925-41be-9887-5176d538c690 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:52.764528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08c915-0925-41be-9887-5176d538c690 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:52.765211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08c915-0925-41be-9887-5176d538c690 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:52.766855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08c915-0925-41be-9887-5176d538c690 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:52.781650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 384/786] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:52.790673 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d962a0b-3845-400d-b6d9-c592122e29c9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:53.053792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d962a0b-3845-400d-b6d9-c592122e29c9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:53.088961 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939abab1-3f9a-4f76-9362-7918b642800a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:53.089589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939abab1-3f9a-4f76-9362-7918b642800a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:53.089866 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939abab1-3f9a-4f76-9362-7918b642800a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:53.090136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939abab1-3f9a-4f76-9362-7918b642800a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:53.097385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939abab1-3f9a-4f76-9362-7918b642800a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:53.098808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939abab1-3f9a-4f76-9362-7918b642800a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-939abab1-3f9a-4f76-9362-7918b642800a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:53.110904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:53.111928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:53.112884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:53.112884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:53.112884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:53.112884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:53.112884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:53.112884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:53.112884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:53.112884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:53.112884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:53.112884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:53.115988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 403/787] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:53.118126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08c915-0925-41be-9887-5176d538c690 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:53.139436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:53.272995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08c915-0925-41be-9887-5176d538c690 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:53.324905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11745ebd-52e7-4791-84e1-b32af26fb0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:53.325460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11745ebd-52e7-4791-84e1-b32af26fb0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:53.325737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11745ebd-52e7-4791-84e1-b32af26fb0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:53.326005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11745ebd-52e7-4791-84e1-b32af26fb0c8 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:53.326763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11745ebd-52e7-4791-84e1-b32af26fb0c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:53.327095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11745ebd-52e7-4791-84e1-b32af26fb0c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:53.328572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11745ebd-52e7-4791-84e1-b32af26fb0c8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:53.358622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 385/788] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 22 16:53:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:53.364865 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939abab1-3f9a-4f76-9362-7918b642800a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:53.453300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd705-3778-4dcd-8cbf-8b4cc69c1aca None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:53.453623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd705-3778-4dcd-8cbf-8b4cc69c1aca None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:53.453998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd705-3778-4dcd-8cbf-8b4cc69c1aca None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:53.455473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 386/789] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:53:53.460413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a370c-cf39-4026-abe3-fc8ea98dd7a8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:53.460927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a370c-cf39-4026-abe3-fc8ea98dd7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:53.461451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a370c-cf39-4026-abe3-fc8ea98dd7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:53.471504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:53.495504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2e8a370c-cf39-4026-abe3-fc8ea98dd7a8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:53.496693 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2e8a370c-cf39-4026-abe3-fc8ea98dd7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:53.529845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939abab1-3f9a-4f76-9362-7918b642800a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:53.566029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80b927-a410-409d-b2cb-256f266b6446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:53.566647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b927-a410-409d-b2cb-256f266b6446 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:53.566920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b927-a410-409d-b2cb-256f266b6446 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:53.567211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b927-a410-409d-b2cb-256f266b6446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:53.590998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80b927-a410-409d-b2cb-256f266b6446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:53.592421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80b927-a410-409d-b2cb-256f266b6446 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-8d80b927-a410-409d-b2cb-256f266b6446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:53.614977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:53.615992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:53.616904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:53.616904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:53.616904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:53.617649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 404/790] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:53.624074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80b927-a410-409d-b2cb-256f266b6446 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:53.636816 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:53.651445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 387/791] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:53:53.751778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80b927-a410-409d-b2cb-256f266b6446 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:53.761978 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:53.794563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdb082-0a62-4b9a-b50a-2ed5cc7a7302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:53.795195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdb082-0a62-4b9a-b50a-2ed5cc7a7302 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:53.795469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdb082-0a62-4b9a-b50a-2ed5cc7a7302 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:53.795736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdb082-0a62-4b9a-b50a-2ed5cc7a7302 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:53.796378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdb082-0a62-4b9a-b50a-2ed5cc7a7302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:53.796674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdb082-0a62-4b9a-b50a-2ed5cc7a7302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:53.798022 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdb082-0a62-4b9a-b50a-2ed5cc7a7302 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:53.815703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 405/792] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:53.821513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a370c-cf39-4026-abe3-fc8ea98dd7a8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:53.953939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a370c-cf39-4026-abe3-fc8ea98dd7a8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:53.991054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090ad8c-86a7-4bb8-9bad-c1017cd51a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:53.991695 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090ad8c-86a7-4bb8-9bad-c1017cd51a03 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:53.991978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090ad8c-86a7-4bb8-9bad-c1017cd51a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:53.992253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090ad8c-86a7-4bb8-9bad-c1017cd51a03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:53.999621 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090ad8c-86a7-4bb8-9bad-c1017cd51a03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:54.001122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090ad8c-86a7-4bb8-9bad-c1017cd51a03 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-9090ad8c-86a7-4bb8-9bad-c1017cd51a03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:54.012890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:54.013825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:54.015270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:54.015270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:54.015270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:54.015270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:54.015270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:54.015270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:54.015270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:54.015270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:54.015270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 388/793] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:54.021622 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdb082-0a62-4b9a-b50a-2ed5cc7a7302 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:54.096459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090ad8c-86a7-4bb8-9bad-c1017cd51a03 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:54.147408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdb082-0a62-4b9a-b50a-2ed5cc7a7302 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:54.183060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ba4fe-c2ac-4a46-b64c-fe109cb9e079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:54.183667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ba4fe-c2ac-4a46-b64c-fe109cb9e079 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:54.183944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ba4fe-c2ac-4a46-b64c-fe109cb9e079 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:54.184214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ba4fe-c2ac-4a46-b64c-fe109cb9e079 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:54.184902 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ba4fe-c2ac-4a46-b64c-fe109cb9e079 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:54.185202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ba4fe-c2ac-4a46-b64c-fe109cb9e079 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:54.186651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ba4fe-c2ac-4a46-b64c-fe109cb9e079 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:54.201786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 406/794] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:54.208583 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ba4fe-c2ac-4a46-b64c-fe109cb9e079 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:54.223488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:54.239902 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:54.325209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090ad8c-86a7-4bb8-9bad-c1017cd51a03 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:54.337555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ba4fe-c2ac-4a46-b64c-fe109cb9e079 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:54.364272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2684d-9578-4c4a-9d79-f5714bbfd17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:54.365015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2684d-9578-4c4a-9d79-f5714bbfd17b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:54.365306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2684d-9578-4c4a-9d79-f5714bbfd17b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:54.365699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2684d-9578-4c4a-9d79-f5714bbfd17b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:54.366521 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2684d-9578-4c4a-9d79-f5714bbfd17b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:54.367926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2684d-9578-4c4a-9d79-f5714bbfd17b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c0d2684d-9578-4c4a-9d79-f5714bbfd17b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:54.378823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:54.379922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:54.381202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 389/795] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Sep 22 16:53:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:54.382005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3a9ca-85b8-4046-9f84-8bdfd38de9a0 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:54.383018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5811d79-1416-4d32-8b91-04946a9e9b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:54.384179 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5811d79-1416-4d32-8b91-04946a9e9b5a None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:54.384947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5811d79-1416-4d32-8b91-04946a9e9b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:54.385603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5811d79-1416-4d32-8b91-04946a9e9b5a None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/b1a54beb68ba4610976524684bc26d5b/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:54.388140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5811d79-1416-4d32-8b91-04946a9e9b5a None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=b1a54beb68ba4610976524684bc26d5b, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:54.388470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3a9ca-85b8-4046-9f84-8bdfd38de9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:54.388841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3a9ca-85b8-4046-9f84-8bdfd38de9a0 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:54.390334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5811d79-1416-4d32-8b91-04946a9e9b5a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:54.394478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 390/796] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:54.394478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3a9ca-85b8-4046-9f84-8bdfd38de9a0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:54.415198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 407/797] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:53:54 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/b1a54beb68ba4610976524684bc26d5b/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:53:54.417590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc66af-1949-4ec6-9154-a9011d141ae2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:54.417911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc66af-1949-4ec6-9154-a9011d141ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:54.418159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc66af-1949-4ec6-9154-a9011d141ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:54.450185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ecdc66af-1949-4ec6-9154-a9011d141ae2 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:54.451459 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ecdc66af-1949-4ec6-9154-a9011d141ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:54.537503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3a9ca-85b8-4046-9f84-8bdfd38de9a0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:54.564423 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:54.565588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 408/798] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:53:54.567086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c2c07-4353-421f-a53f-eebb77b31a38 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:54.567086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c2c07-4353-421f-a53f-eebb77b31a38 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:54.567086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c2c07-4353-421f-a53f-eebb77b31a38 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:54.572564 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 409/799] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:54.574328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2927c79-d9d7-48b7-aec7-b8bd50af4975 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:54.574586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2927c79-d9d7-48b7-aec7-b8bd50af4975 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:54.574863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2927c79-d9d7-48b7-aec7-b8bd50af4975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:54.585336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24950be-9d32-4049-abba-115372448c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:54.586318 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24950be-9d32-4049-abba-115372448c17 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:54.586817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24950be-9d32-4049-abba-115372448c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:54.587914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24950be-9d32-4049-abba-115372448c17 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:54.588869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24950be-9d32-4049-abba-115372448c17 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:54.589516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24950be-9d32-4049-abba-115372448c17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:54.591291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24950be-9d32-4049-abba-115372448c17 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:54.593788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e2927c79-d9d7-48b7-aec7-b8bd50af4975 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:54.595190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e2927c79-d9d7-48b7-aec7-b8bd50af4975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:54.597460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 391/800] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 22 16:53:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:54.603432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24950be-9d32-4049-abba-115372448c17 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:54.609264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:54.676961 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 410/801] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:53:54.715717 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24950be-9d32-4049-abba-115372448c17 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:54.755234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e3d7a-e375-4add-85e7-49dd86eb4298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:54.756160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e3d7a-e375-4add-85e7-49dd86eb4298 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:54.756644 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e3d7a-e375-4add-85e7-49dd86eb4298 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:54.757093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e3d7a-e375-4add-85e7-49dd86eb4298 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:54.757996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e3d7a-e375-4add-85e7-49dd86eb4298 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:54.758478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e3d7a-e375-4add-85e7-49dd86eb4298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:54.760218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e3d7a-e375-4add-85e7-49dd86eb4298 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:54.766266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 392/802] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 22 16:53:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:54.772041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2927c79-d9d7-48b7-aec7-b8bd50af4975 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:54.897805 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2927c79-d9d7-48b7-aec7-b8bd50af4975 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:54.921917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e3d7a-e375-4add-85e7-49dd86eb4298 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:54.935922 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad3ca3-3b83-4389-bb18-23e47c500740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:54.936620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad3ca3-3b83-4389-bb18-23e47c500740 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:54.936906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad3ca3-3b83-4389-bb18-23e47c500740 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:54.937209 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad3ca3-3b83-4389-bb18-23e47c500740 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:54.938226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad3ca3-3b83-4389-bb18-23e47c500740 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:54.939877 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad3ca3-3b83-4389-bb18-23e47c500740 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:54.970602 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 411/803] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 16:53:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:53:54.994016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:55.008769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad3ca3-3b83-4389-bb18-23e47c500740 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:55.011584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:55.158401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e3d7a-e375-4add-85e7-49dd86eb4298 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:55.162557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad3ca3-3b83-4389-bb18-23e47c500740 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:55.208890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38025d20-f03f-44fd-bbe0-92f2bdafaf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:55.209125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58804a5b-e86b-4e17-97c4-d9754d4fe9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:55.209533 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38025d20-f03f-44fd-bbe0-92f2bdafaf13 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:55.209705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58804a5b-e86b-4e17-97c4-d9754d4fe9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:55.209799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38025d20-f03f-44fd-bbe0-92f2bdafaf13 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:55.209955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58804a5b-e86b-4e17-97c4-d9754d4fe9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:55.210056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38025d20-f03f-44fd-bbe0-92f2bdafaf13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:55.210217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58804a5b-e86b-4e17-97c4-d9754d4fe9a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:55.222245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58804a5b-e86b-4e17-97c4-d9754d4fe9a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:55.222429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38025d20-f03f-44fd-bbe0-92f2bdafaf13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-38025d20-f03f-44fd-bbe0-92f2bdafaf13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:55.227789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:55.228744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:53:55.230728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-58804a5b-e86b-4e17-97c4-d9754d4fe9a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:55.231618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:55.232661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:55.236426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:55.237790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 412/804] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:53:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:55.237919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 393/805] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:55.238042 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38025d20-f03f-44fd-bbe0-92f2bdafaf13 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:55.238979 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58804a5b-e86b-4e17-97c4-d9754d4fe9a0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:55.313441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:55.357840 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58804a5b-e86b-4e17-97c4-d9754d4fe9a0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:55.365510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38025d20-f03f-44fd-bbe0-92f2bdafaf13 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:55.376619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:55.408115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8e155-2ebc-44df-8373-b1126a2829f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:55.408744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8e155-2ebc-44df-8373-b1126a2829f6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:55.408967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8e155-2ebc-44df-8373-b1126a2829f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:55.409241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8e155-2ebc-44df-8373-b1126a2829f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:55.409912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8e155-2ebc-44df-8373-b1126a2829f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:55.410214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8e155-2ebc-44df-8373-b1126a2829f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:55.411706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8e155-2ebc-44df-8373-b1126a2829f6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:55.412305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7aca6-5e68-4ef6-9926-2278e3835050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:55.413225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7aca6-5e68-4ef6-9926-2278e3835050 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:55.413726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7aca6-5e68-4ef6-9926-2278e3835050 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:55.414263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7aca6-5e68-4ef6-9926-2278e3835050 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:55.415133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7aca6-5e68-4ef6-9926-2278e3835050 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:55.415624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7aca6-5e68-4ef6-9926-2278e3835050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:55.417847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7aca6-5e68-4ef6-9926-2278e3835050 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:55.421202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 413/806] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:53:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:55.428024 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8e155-2ebc-44df-8373-b1126a2829f6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:55.429064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 394/807] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:55.436422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7aca6-5e68-4ef6-9926-2278e3835050 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:55.595084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8e155-2ebc-44df-8373-b1126a2829f6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:55.622356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7aca6-5e68-4ef6-9926-2278e3835050 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:55.624173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:55.645628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872da7f2-0f46-41be-bacd-80c8c3dd5c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:55.646616 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872da7f2-0f46-41be-bacd-80c8c3dd5c83 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:55.647166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872da7f2-0f46-41be-bacd-80c8c3dd5c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:55.648139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872da7f2-0f46-41be-bacd-80c8c3dd5c83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:55.663608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872da7f2-0f46-41be-bacd-80c8c3dd5c83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:55.663789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d2086-9607-459b-879f-2ebf6a92fa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:55.664214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d2086-9607-459b-879f-2ebf6a92fa4a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:55.664493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d2086-9607-459b-879f-2ebf6a92fa4a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:55.664768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d2086-9607-459b-879f-2ebf6a92fa4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:55.666535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872da7f2-0f46-41be-bacd-80c8c3dd5c83 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:55.680097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d2086-9607-459b-879f-2ebf6a92fa4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:55.682506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d2086-9607-459b-879f-2ebf6a92fa4a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-872da7f2-0f46-41be-bacd-80c8c3dd5c83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:55.688540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:55.689519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:55.690408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:55.690408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:55.690408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:55.693740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 414/808] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:55.698853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d61498-067b-46e0-8c8d-97cad7fc4dd3 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:55.698853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d61498-067b-46e0-8c8d-97cad7fc4dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:55.698853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d61498-067b-46e0-8c8d-97cad7fc4dd3 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:55.699613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 415/809] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:55.700260 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d61498-067b-46e0-8c8d-97cad7fc4dd3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-678d2086-9607-459b-879f-2ebf6a92fa4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:55.712359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:55.713921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:55.714833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:55.714833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:55.714833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:55.719429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 395/810] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:53:55.720867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3de01b-d85c-42b2-98e0-bdf0c3f312ec None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:55.721154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3de01b-d85c-42b2-98e0-bdf0c3f312ec None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:55.721429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3de01b-d85c-42b2-98e0-bdf0c3f312ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:55.750390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2f3de01b-d85c-42b2-98e0-bdf0c3f312ec None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:55.752157 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2f3de01b-d85c-42b2-98e0-bdf0c3f312ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:55.785698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:55.849596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d61498-067b-46e0-8c8d-97cad7fc4dd3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:55.888857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf73426-5191-443d-b46d-aa1854943729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:55.889490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf73426-5191-443d-b46d-aa1854943729 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:55.889770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf73426-5191-443d-b46d-aa1854943729 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:55.890046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf73426-5191-443d-b46d-aa1854943729 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:55.890700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf73426-5191-443d-b46d-aa1854943729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:55.891050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf73426-5191-443d-b46d-aa1854943729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:55.892446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf73426-5191-443d-b46d-aa1854943729 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:55.906525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 416/811] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:55.907530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf73426-5191-443d-b46d-aa1854943729 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:55.943159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:55.973421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 396/812] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:53:55.974577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3de01b-d85c-42b2-98e0-bdf0c3f312ec None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:56.049508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf73426-5191-443d-b46d-aa1854943729 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:56.090892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3de01b-d85c-42b2-98e0-bdf0c3f312ec None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:56.131114 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e39163-a089-4141-83a4-6b2ecb2d9c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:56.132274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e39163-a089-4141-83a4-6b2ecb2d9c97 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:56.132743 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e39163-a089-4141-83a4-6b2ecb2d9c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:56.133361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e39163-a089-4141-83a4-6b2ecb2d9c97 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:56.134977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53210f14-ee8b-4868-92d8-e90a9f24ef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:56.135732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53210f14-ee8b-4868-92d8-e90a9f24ef0a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:56.136151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53210f14-ee8b-4868-92d8-e90a9f24ef0a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:56.136674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53210f14-ee8b-4868-92d8-e90a9f24ef0a None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:56.137811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53210f14-ee8b-4868-92d8-e90a9f24ef0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:56.137944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53210f14-ee8b-4868-92d8-e90a9f24ef0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:56.139444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53210f14-ee8b-4868-92d8-e90a9f24ef0a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:56.143326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e39163-a089-4141-83a4-6b2ecb2d9c97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:56.144985 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e39163-a089-4141-83a4-6b2ecb2d9c97 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:56.157695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 417/813] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:56.162997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53210f14-ee8b-4868-92d8-e90a9f24ef0a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-a1e39163-a089-4141-83a4-6b2ecb2d9c97 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:56.168944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:56.170062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:56.171275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:56.171275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:56.171275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:56.171275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:56.171275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:56.171275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:56.171275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:56.171275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:56.171275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:53:56.171275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:56.173472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 397/814] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 22 16:53:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:56.183757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e39163-a089-4141-83a4-6b2ecb2d9c97 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:56.208897 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:56.339095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e39163-a089-4141-83a4-6b2ecb2d9c97 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:56.341317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:56.361089 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53210f14-ee8b-4868-92d8-e90a9f24ef0a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:56.380061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f365222-83f1-49cc-8e7e-1f6dbb473ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:56.381816 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365222-83f1-49cc-8e7e-1f6dbb473ddc None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:56.382328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365222-83f1-49cc-8e7e-1f6dbb473ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:56.382801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365222-83f1-49cc-8e7e-1f6dbb473ddc None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:56.383761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f365222-83f1-49cc-8e7e-1f6dbb473ddc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:56.384375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f365222-83f1-49cc-8e7e-1f6dbb473ddc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:56.386065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f365222-83f1-49cc-8e7e-1f6dbb473ddc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:56.407542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 398/815] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 22 16:53:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:56.409003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f365222-83f1-49cc-8e7e-1f6dbb473ddc None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:56.426675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c09e9-13ad-43e4-8102-3bfc2350f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:56.427777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c09e9-13ad-43e4-8102-3bfc2350f628 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:56.428261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c09e9-13ad-43e4-8102-3bfc2350f628 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:56.428759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c09e9-13ad-43e4-8102-3bfc2350f628 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:56.441796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c09e9-13ad-43e4-8102-3bfc2350f628 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:56.443853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c09e9-13ad-43e4-8102-3bfc2350f628 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-5f7c09e9-13ad-43e4-8102-3bfc2350f628 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:56.457774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:56.458767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:56.459926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:56.459926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:56.459926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:56.459926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:56.459926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:56.459926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:56.459926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:56.459926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:56.459926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:53:56.459926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:56.462635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 418/816] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 22 16:53:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:56.468567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c09e9-13ad-43e4-8102-3bfc2350f628 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:56.495289 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:56.529699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f365222-83f1-49cc-8e7e-1f6dbb473ddc None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:56.566405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a341407-a1ca-4998-812e-0d24dcd02536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:56.567344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a341407-a1ca-4998-812e-0d24dcd02536 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:56.567766 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a341407-a1ca-4998-812e-0d24dcd02536 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:56.568179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a341407-a1ca-4998-812e-0d24dcd02536 None admin] PATH_INFO: `/v3/services/network` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:56.568900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a341407-a1ca-4998-812e-0d24dcd02536 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:56.570477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a341407-a1ca-4998-812e-0d24dcd02536 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-7a341407-a1ca-4998-812e-0d24dcd02536 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:56.580573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:56.581797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 16:53:56.582845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:56.583788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c09e9-13ad-43e4-8102-3bfc2350f628 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:56.590005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 399/817] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:56 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:56.591130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a341407-a1ca-4998-812e-0d24dcd02536 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:56.622293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61fa8-5f85-4412-99ee-89196bdc1856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:56.623119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61fa8-5f85-4412-99ee-89196bdc1856 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:56.623670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61fa8-5f85-4412-99ee-89196bdc1856 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:56.623856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61fa8-5f85-4412-99ee-89196bdc1856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:56.636559 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a61fa8-5f85-4412-99ee-89196bdc1856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:56.637930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a61fa8-5f85-4412-99ee-89196bdc1856 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-61a61fa8-5f85-4412-99ee-89196bdc1856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:56.653416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:56.654631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:56.655757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:56.655757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:56.655757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:56.661587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 419/818] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:56.661587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61fa8-5f85-4412-99ee-89196bdc1856 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:56.665215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:56.709862 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a341407-a1ca-4998-812e-0d24dcd02536 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:56.745872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9d631-88f9-4a41-bd1d-c08eb06cb78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:56.746721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d631-88f9-4a41-bd1d-c08eb06cb78b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:56.747170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d631-88f9-4a41-bd1d-c08eb06cb78b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:56.747660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d631-88f9-4a41-bd1d-c08eb06cb78b None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:56.749047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9d631-88f9-4a41-bd1d-c08eb06cb78b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:56.749594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9d631-88f9-4a41-bd1d-c08eb06cb78b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:56.751359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9d631-88f9-4a41-bd1d-c08eb06cb78b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:56.755754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:56.782038 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 400/819] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 22 16:53:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:56.783293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9d631-88f9-4a41-bd1d-c08eb06cb78b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:56.787392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61fa8-5f85-4412-99ee-89196bdc1856 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:56.830399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e309448-9fee-494b-ac44-cdbb7b3bd0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:56.831083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e309448-9fee-494b-ac44-cdbb7b3bd0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:56.831419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e309448-9fee-494b-ac44-cdbb7b3bd0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:56.831694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e309448-9fee-494b-ac44-cdbb7b3bd0c4 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:56.832721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e309448-9fee-494b-ac44-cdbb7b3bd0c4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:56.832721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e309448-9fee-494b-ac44-cdbb7b3bd0c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:56.834024 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e309448-9fee-494b-ac44-cdbb7b3bd0c4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:56.843692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 420/820] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:53:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:56.843692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e309448-9fee-494b-ac44-cdbb7b3bd0c4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:56.900249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9d631-88f9-4a41-bd1d-c08eb06cb78b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:56.917518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:56.939980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445b390-e22c-4655-a329-992445b896b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:56.941014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445b390-e22c-4655-a329-992445b896b6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:56.941504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445b390-e22c-4655-a329-992445b896b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:56.942019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445b390-e22c-4655-a329-992445b896b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:56.942900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445b390-e22c-4655-a329-992445b896b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:56.943402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445b390-e22c-4655-a329-992445b896b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:56.945037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445b390-e22c-4655-a329-992445b896b6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:56.962710 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e309448-9fee-494b-ac44-cdbb7b3bd0c4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:56.963045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 401/821] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:56.964266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445b390-e22c-4655-a329-992445b896b6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:57.002829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34625022-42f3-47be-aa28-648206509015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:57.003448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34625022-42f3-47be-aa28-648206509015 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.003775 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34625022-42f3-47be-aa28-648206509015 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.004053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34625022-42f3-47be-aa28-648206509015 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.023206 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:57.077202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445b390-e22c-4655-a329-992445b896b6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:57.081235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34625022-42f3-47be-aa28-648206509015 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:57.082797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34625022-42f3-47be-aa28-648206509015 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:57.126898 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb00aa1-60af-4126-8522-443822005da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:57.127982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb00aa1-60af-4126-8522-443822005da1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.128504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb00aa1-60af-4126-8522-443822005da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.129006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb00aa1-60af-4126-8522-443822005da1 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.129669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb00aa1-60af-4126-8522-443822005da1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:57.129973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb00aa1-60af-4126-8522-443822005da1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:57.131499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb00aa1-60af-4126-8522-443822005da1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:57.137767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 402/822] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:53:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:57.139947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb00aa1-60af-4126-8522-443822005da1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:57.172386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:57.212471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 421/823] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:53:57.213979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a740d-338d-47fb-93f3-f2e9c4a59929 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.214333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a740d-338d-47fb-93f3-f2e9c4a59929 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.214695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a740d-338d-47fb-93f3-f2e9c4a59929 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.218987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 422/824] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:57.218987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a740d-338d-47fb-93f3-f2e9c4a59929 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:57.260929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb00aa1-60af-4126-8522-443822005da1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:57.277083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:57.296834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4489f-0dfa-4b22-9243-a6194e92ce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:57.297470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4489f-0dfa-4b22-9243-a6194e92ce73 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.297754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4489f-0dfa-4b22-9243-a6194e92ce73 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.298028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4489f-0dfa-4b22-9243-a6194e92ce73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.305169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4489f-0dfa-4b22-9243-a6194e92ce73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:57.307528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4489f-0dfa-4b22-9243-a6194e92ce73 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-61d4489f-0dfa-4b22-9243-a6194e92ce73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:57.319069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:57.319992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:57.320858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:57.320858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:57.320858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:57.320858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:57.320858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:57.320858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:57.320858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:57.320858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:57.320858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 403/825] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:57.320858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4489f-0dfa-4b22-9243-a6194e92ce73 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:57.342676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a740d-338d-47fb-93f3-f2e9c4a59929 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:57.380624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747058b-7895-4fdc-b558-a07c38d865df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:57.381234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747058b-7895-4fdc-b558-a07c38d865df None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.381552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747058b-7895-4fdc-b558-a07c38d865df None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.381779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747058b-7895-4fdc-b558-a07c38d865df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.382445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747058b-7895-4fdc-b558-a07c38d865df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:57.383817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747058b-7895-4fdc-b558-a07c38d865df None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:57.406338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 423/826] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 16:53:57 2022] POST /identity/v3/endpoints => generated 337 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:53:57.407984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b71a9-6bfe-4a0e-86c1-a4db0a06b287 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.408296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b71a9-6bfe-4a0e-86c1-a4db0a06b287 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.408554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b71a9-6bfe-4a0e-86c1-a4db0a06b287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.439970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-038b71a9-6bfe-4a0e-86c1-a4db0a06b287 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:57.442063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-038b71a9-6bfe-4a0e-86c1-a4db0a06b287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:57.444140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4489f-0dfa-4b22-9243-a6194e92ce73 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:57.466241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:57.501591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f2e34-28d9-4d26-b66a-113e603febef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:57.502755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f2e34-28d9-4d26-b66a-113e603febef None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.503295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f2e34-28d9-4d26-b66a-113e603febef None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.503785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f2e34-28d9-4d26-b66a-113e603febef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.518055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f2e34-28d9-4d26-b66a-113e603febef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:57.520109 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f2e34-28d9-4d26-b66a-113e603febef None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-580f2e34-28d9-4d26-b66a-113e603febef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:57.538261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:57.539313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:57.540208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:57.540208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:57.540208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:57.543397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 404/827] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:57.544343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f2e34-28d9-4d26-b66a-113e603febef None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:57.551228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 424/828] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:53:57.552537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b71a9-6bfe-4a0e-86c1-a4db0a06b287 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:57.582630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:57.687092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f2e34-28d9-4d26-b66a-113e603febef None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:57.713070 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b71a9-6bfe-4a0e-86c1-a4db0a06b287 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:57.728071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d874d-c17a-4998-8cb8-b4b0db14a27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:57.728679 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d874d-c17a-4998-8cb8-b4b0db14a27e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.729477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d874d-c17a-4998-8cb8-b4b0db14a27e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.729746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d874d-c17a-4998-8cb8-b4b0db14a27e None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.730431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d874d-c17a-4998-8cb8-b4b0db14a27e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:57.730729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d874d-c17a-4998-8cb8-b4b0db14a27e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:57.732189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d874d-c17a-4998-8cb8-b4b0db14a27e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:57.744937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 405/829] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:57.746280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70208c00-8702-403d-98a4-73b44c6f80ab None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.746565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70208c00-8702-403d-98a4-73b44c6f80ab None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.746831 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70208c00-8702-403d-98a4-73b44c6f80ab None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.748176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 406/830] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:53:57.752833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70208c00-8702-403d-98a4-73b44c6f80ab None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:57.761623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86f2d8-a3e1-4e02-b72f-dff0a74f43b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:57.763063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f2d8-a3e1-4e02-b72f-dff0a74f43b5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.763912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f2d8-a3e1-4e02-b72f-dff0a74f43b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.763912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f2d8-a3e1-4e02-b72f-dff0a74f43b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.764926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f2d8-a3e1-4e02-b72f-dff0a74f43b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:57.765302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f2d8-a3e1-4e02-b72f-dff0a74f43b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:57.767112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f2d8-a3e1-4e02-b72f-dff0a74f43b5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:57.779589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 425/831] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:57.784993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27755656-7cf5-4144-871b-ff0de8bd7012 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.785278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27755656-7cf5-4144-871b-ff0de8bd7012 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.785556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27755656-7cf5-4144-871b-ff0de8bd7012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.803832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-27755656-7cf5-4144-871b-ff0de8bd7012 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:53:57.805020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-27755656-7cf5-4144-871b-ff0de8bd7012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:53:57.815557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:57.849035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:57.866542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70208c00-8702-403d-98a4-73b44c6f80ab None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:57.877200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 426/832] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:53:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:53:57.878048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27755656-7cf5-4144-871b-ff0de8bd7012 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:57.904835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a1d55-4816-4334-a98a-7a304de21d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:57.905561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a1d55-4816-4334-a98a-7a304de21d1d None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:57.905847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a1d55-4816-4334-a98a-7a304de21d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:57.906126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a1d55-4816-4334-a98a-7a304de21d1d None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/abbbe2b3f23b4c1caf8e20825cc2cfee/roles/6ae0df226be44eccbaa70e0140db1c40` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:57.923199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a1d55-4816-4334-a98a-7a304de21d1d None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=abbbe2b3f23b4c1caf8e20825cc2cfee, role_id=6ae0df226be44eccbaa70e0140db1c40)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:57.924924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a1d55-4816-4334-a98a-7a304de21d1d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:57.946981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 407/833] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:53:57 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/abbbe2b3f23b4c1caf8e20825cc2cfee/roles/6ae0df226be44eccbaa70e0140db1c40 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:53:57.946981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a1d55-4816-4334-a98a-7a304de21d1d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:58.007474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27755656-7cf5-4144-871b-ff0de8bd7012 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:58.052176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b4132-ee78-414f-86b4-3adcf93c0aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:58.053031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b4132-ee78-414f-86b4-3adcf93c0aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:58.053762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b4132-ee78-414f-86b4-3adcf93c0aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:58.054040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b4132-ee78-414f-86b4-3adcf93c0aa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:58.056517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a1d55-4816-4334-a98a-7a304de21d1d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:58.064090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b4132-ee78-414f-86b4-3adcf93c0aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:58.066460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b4132-ee78-414f-86b4-3adcf93c0aa6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-6c4b4132-ee78-414f-86b4-3adcf93c0aa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:53:58.084063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:58.085060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:58.085959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:53:58.085959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:53:58.085959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:53:58.085959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:53:58.085959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:53:58.085959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:53:58.085959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:53:58.085959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:58.087281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 427/834] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:58.093083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b4132-ee78-414f-86b4-3adcf93c0aa6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:58.099073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:58.106363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e7237-34ea-4881-a571-b94c06992b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:58.107076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7237-34ea-4881-a571-b94c06992b21 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:58.107361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7237-34ea-4881-a571-b94c06992b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:58.107634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7237-34ea-4881-a571-b94c06992b21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:58.111977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:58.125754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e7237-34ea-4881-a571-b94c06992b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-0d1e7237-34ea-4881-a571-b94c06992b21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:53:58.136922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:58.138140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:53:58.139085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:58.139872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 408/835] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:58.145312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e7237-34ea-4881-a571-b94c06992b21 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:58.229876 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b4132-ee78-414f-86b4-3adcf93c0aa6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:58.268904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49e3d1-4689-4b2d-8a07-66d59af5623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:58.269152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e7237-34ea-4881-a571-b94c06992b21 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:58.269521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49e3d1-4689-4b2d-8a07-66d59af5623a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:58.269802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49e3d1-4689-4b2d-8a07-66d59af5623a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:58.270076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49e3d1-4689-4b2d-8a07-66d59af5623a None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:58.270895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49e3d1-4689-4b2d-8a07-66d59af5623a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:58.271346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49e3d1-4689-4b2d-8a07-66d59af5623a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:58.272768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49e3d1-4689-4b2d-8a07-66d59af5623a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:58.291289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 428/836] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:53:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:58.293153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49e3d1-4689-4b2d-8a07-66d59af5623a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:58.306314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc050f0d-6d37-415e-8893-cc37142880ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:58.306851 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc050f0d-6d37-415e-8893-cc37142880ec None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:58.307723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc050f0d-6d37-415e-8893-cc37142880ec None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:58.307998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc050f0d-6d37-415e-8893-cc37142880ec None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:58.308655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc050f0d-6d37-415e-8893-cc37142880ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:58.308960 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc050f0d-6d37-415e-8893-cc37142880ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:58.310329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc050f0d-6d37-415e-8893-cc37142880ec None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:58.318875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 409/837] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:53:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:58.319719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc050f0d-6d37-415e-8893-cc37142880ec None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:58.368121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:58.381460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:58.412044 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49e3d1-4689-4b2d-8a07-66d59af5623a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:58.427518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc050f0d-6d37-415e-8893-cc37142880ec None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:58.450310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e3bdb-914a-432d-a6f5-b17df67f9a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:58.451075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e3bdb-914a-432d-a6f5-b17df67f9a28 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:58.451375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e3bdb-914a-432d-a6f5-b17df67f9a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:58.451729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e3bdb-914a-432d-a6f5-b17df67f9a28 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:58.452305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e3bdb-914a-432d-a6f5-b17df67f9a28 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:58.453828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e3bdb-914a-432d-a6f5-b17df67f9a28 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-bf5e3bdb-914a-432d-a6f5-b17df67f9a28 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:58.463039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:58.464142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:58.465016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 429/838] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:53:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:58.465816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e3bdb-914a-432d-a6f5-b17df67f9a28 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:58.466796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7524ca-c0ed-4c2f-ac3f-3ef93b260582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:58.467444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7524ca-c0ed-4c2f-ac3f-3ef93b260582 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:58.467720 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7524ca-c0ed-4c2f-ac3f-3ef93b260582 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:58.467986 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7524ca-c0ed-4c2f-ac3f-3ef93b260582 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:58.468705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7524ca-c0ed-4c2f-ac3f-3ef93b260582 None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=f6c1ce47d8704111988a78e625fd968c) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:58.469010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7524ca-c0ed-4c2f-ac3f-3ef93b260582 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:58.470436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7524ca-c0ed-4c2f-ac3f-3ef93b260582 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:58.483133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 410/839] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:53:58 2022] GET /identity/v3/role_assignments?user.id=f6c1ce47d8704111988a78e625fd968c&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:58.483133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7524ca-c0ed-4c2f-ac3f-3ef93b260582 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:58.612850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7524ca-c0ed-4c2f-ac3f-3ef93b260582 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:58.623951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e3bdb-914a-432d-a6f5-b17df67f9a28 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:58.641275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:58.661916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d367-244a-41b5-8623-04d7a090f56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:58.662554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d367-244a-41b5-8623-04d7a090f56d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:58.662834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d367-244a-41b5-8623-04d7a090f56d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:58.663653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d367-244a-41b5-8623-04d7a090f56d None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:58.664316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373d367-244a-41b5-8623-04d7a090f56d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:58.664624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373d367-244a-41b5-8623-04d7a090f56d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:58.666726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373d367-244a-41b5-8623-04d7a090f56d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:58.668814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:58.680018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92545bc-aefd-4520-8e10-ddd201c90fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:58.681031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92545bc-aefd-4520-8e10-ddd201c90fae None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:58.681163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 411/840] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:53:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:58.681306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92545bc-aefd-4520-8e10-ddd201c90fae None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:58.681577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92545bc-aefd-4520-8e10-ddd201c90fae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:58.682389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d367-244a-41b5-8623-04d7a090f56d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:58.697415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92545bc-aefd-4520-8e10-ddd201c90fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:58.699320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92545bc-aefd-4520-8e10-ddd201c90fae None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e92545bc-aefd-4520-8e10-ddd201c90fae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:58.715310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:58.716785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:58.717668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:58.717668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:58.717668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:58.717668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 430/841] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:58.722560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92545bc-aefd-4520-8e10-ddd201c90fae None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:58.824275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d367-244a-41b5-8623-04d7a090f56d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:58.863224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92545bc-aefd-4520-8e10-ddd201c90fae None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:58.873114 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296c0c2-97c8-4ce8-9177-99d91a1fb4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:58.873784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296c0c2-97c8-4ce8-9177-99d91a1fb4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:58.874071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296c0c2-97c8-4ce8-9177-99d91a1fb4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:58.874349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296c0c2-97c8-4ce8-9177-99d91a1fb4e7 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:58.875065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296c0c2-97c8-4ce8-9177-99d91a1fb4e7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:58.875374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296c0c2-97c8-4ce8-9177-99d91a1fb4e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:58.877436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296c0c2-97c8-4ce8-9177-99d91a1fb4e7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:58.889422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 412/842] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:53:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:53:58.910168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888b51c-30ac-42b8-812f-45bbbc6f7e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:58.910796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888b51c-30ac-42b8-812f-45bbbc6f7e37 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:58.911128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888b51c-30ac-42b8-812f-45bbbc6f7e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:58.911408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888b51c-30ac-42b8-812f-45bbbc6f7e37 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:58.912076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888b51c-30ac-42b8-812f-45bbbc6f7e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:58.912385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888b51c-30ac-42b8-812f-45bbbc6f7e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:58.913968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888b51c-30ac-42b8-812f-45bbbc6f7e37 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:58.933383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 431/843] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:58.934837 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296c0c2-97c8-4ce8-9177-99d91a1fb4e7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:58.995331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:59.058635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296c0c2-97c8-4ce8-9177-99d91a1fb4e7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:59.098374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde92fa-56ab-4fff-9a05-cd89f4a01b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:59.099240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde92fa-56ab-4fff-9a05-cd89f4a01b5a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:59.099623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde92fa-56ab-4fff-9a05-cd89f4a01b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:59.099986 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde92fa-56ab-4fff-9a05-cd89f4a01b5a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:59.109541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde92fa-56ab-4fff-9a05-cd89f4a01b5a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:59.111140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde92fa-56ab-4fff-9a05-cd89f4a01b5a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-dcde92fa-56ab-4fff-9a05-cd89f4a01b5a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:53:59.123458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:59.124612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:59.125888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:59.125888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:59.125888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:53:59.125888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:53:59.125888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:53:59.125888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:53:59.125888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:53:59.125888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:53:59.125888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 16:53:59.125888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:53:59.127889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 413/844] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:53:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:59.133679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888b51c-30ac-42b8-812f-45bbbc6f7e37 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:59.178646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:59.240136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888b51c-30ac-42b8-812f-45bbbc6f7e37 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:59.276691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cbc57-01b1-413b-b99c-199cce426541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:59.277304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cbc57-01b1-413b-b99c-199cce426541 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:59.277582 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cbc57-01b1-413b-b99c-199cce426541 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:59.277855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cbc57-01b1-413b-b99c-199cce426541 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:59.278593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cbc57-01b1-413b-b99c-199cce426541 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:59.278897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cbc57-01b1-413b-b99c-199cce426541 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:59.280360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cbc57-01b1-413b-b99c-199cce426541 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:59.299242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 432/845] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 22 16:53:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:53:59.305023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde92fa-56ab-4fff-9a05-cd89f4a01b5a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:59.418661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde92fa-56ab-4fff-9a05-cd89f4a01b5a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:59.428219 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:59.455358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a85b2-54b1-47dc-94c0-0ac34baea5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:59.455966 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a85b2-54b1-47dc-94c0-0ac34baea5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:59.456253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a85b2-54b1-47dc-94c0-0ac34baea5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:59.456526 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a85b2-54b1-47dc-94c0-0ac34baea5b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:59.539054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a85b2-54b1-47dc-94c0-0ac34baea5b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:59.542031 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a85b2-54b1-47dc-94c0-0ac34baea5b8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:59.645706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 414/846] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:53:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:53:59.651959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cbc57-01b1-413b-b99c-199cce426541 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:59.757739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:59.766675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cbc57-01b1-413b-b99c-199cce426541 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:59.803028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1661f91-975d-4059-b0d1-7219fef71e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:59.803658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1661f91-975d-4059-b0d1-7219fef71e4a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:59.803928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1661f91-975d-4059-b0d1-7219fef71e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:59.804193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1661f91-975d-4059-b0d1-7219fef71e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:59.815647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1661f91-975d-4059-b0d1-7219fef71e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:59.817091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1661f91-975d-4059-b0d1-7219fef71e4a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-d1661f91-975d-4059-b0d1-7219fef71e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:53:59.834276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:53:59.835405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:53:59.836379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:53:59.836379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:53:59.836379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:53:59.836379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 433/847] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:53:59.842889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a85b2-54b1-47dc-94c0-0ac34baea5b8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:53:59.854467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:53:59.957287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a85b2-54b1-47dc-94c0-0ac34baea5b8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:53:59.993367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a388c-f3d5-4195-9e6a-89476ddf94b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:53:59.993968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a388c-f3d5-4195-9e6a-89476ddf94b3 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:53:59.994237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a388c-f3d5-4195-9e6a-89476ddf94b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:53:59.994617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a388c-f3d5-4195-9e6a-89476ddf94b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:53:59.995283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a388c-f3d5-4195-9e6a-89476ddf94b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:53:59.995580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a388c-f3d5-4195-9e6a-89476ddf94b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:53:59.996923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a388c-f3d5-4195-9e6a-89476ddf94b3 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:00.008971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 415/848] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:00.014461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1661f91-975d-4059-b0d1-7219fef71e4a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:00.126136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1661f91-975d-4059-b0d1-7219fef71e4a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:00.160806 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b11b60-6dbc-4c1f-bdab-64ca11cb2fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:00.161428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11b60-6dbc-4c1f-bdab-64ca11cb2fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:00.161702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11b60-6dbc-4c1f-bdab-64ca11cb2fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:00.161973 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11b60-6dbc-4c1f-bdab-64ca11cb2fd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:00.166151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212deb4e-113a-48e2-8d31-181ee91bba89 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:00.166504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212deb4e-113a-48e2-8d31-181ee91bba89 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:00.166782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212deb4e-113a-48e2-8d31-181ee91bba89 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:00.168352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 416/849] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:00.169416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b11b60-6dbc-4c1f-bdab-64ca11cb2fd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:00.170920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b11b60-6dbc-4c1f-bdab-64ca11cb2fd0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:00.173820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edc04c-0298-43a3-8939-a5f123c0ec61 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:00.174104 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edc04c-0298-43a3-8939-a5f123c0ec61 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:00.174369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edc04c-0298-43a3-8939-a5f123c0ec61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-86b11b60-6dbc-4c1f-bdab-64ca11cb2fd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:54:00.183623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:00.184563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:00.185462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:54:00.185462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:54:00.185462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:54:00.185462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:54:00.185462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:54:00.185462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:54:00.185462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:00.185462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:00.185462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 434/850] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:00.192647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b11b60-6dbc-4c1f-bdab-64ca11cb2fd0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:00.193458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-23edc04c-0298-43a3-8939-a5f123c0ec61 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:00.194633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-23edc04c-0298-43a3-8939-a5f123c0ec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:00.207107 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:00.267100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:00.273872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 417/851] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:00.310590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b11b60-6dbc-4c1f-bdab-64ca11cb2fd0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:00.347800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f5415-c44b-46fd-bacd-34d9e688f66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:00.348800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f5415-c44b-46fd-bacd-34d9e688f66b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:00.349298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f5415-c44b-46fd-bacd-34d9e688f66b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:00.349772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f5415-c44b-46fd-bacd-34d9e688f66b None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:00.350770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f5415-c44b-46fd-bacd-34d9e688f66b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:00.351305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f5415-c44b-46fd-bacd-34d9e688f66b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:00.352927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f5415-c44b-46fd-bacd-34d9e688f66b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:00.371406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 435/852] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:54:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:00.374531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edc04c-0298-43a3-8939-a5f123c0ec61 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:00.479592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edc04c-0298-43a3-8939-a5f123c0ec61 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:00.517917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08e1d8-7480-471b-bbe4-55992cf16fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:00.518444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08e1d8-7480-471b-bbe4-55992cf16fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:00.518725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08e1d8-7480-471b-bbe4-55992cf16fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:00.519024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08e1d8-7480-471b-bbe4-55992cf16fe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:00.519744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08e1d8-7480-471b-bbe4-55992cf16fe4 None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=b1a54beb68ba4610976524684bc26d5b) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:00.520049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08e1d8-7480-471b-bbe4-55992cf16fe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:00.521482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08e1d8-7480-471b-bbe4-55992cf16fe4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:00.528196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 418/853] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:54:00 2022] GET /identity/v3/role_assignments?user.id=b1a54beb68ba4610976524684bc26d5b&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:00.564725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f5415-c44b-46fd-bacd-34d9e688f66b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:00.649269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:00.685867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f5415-c44b-46fd-bacd-34d9e688f66b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:00.729101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76c356-f03a-45de-b999-2b4fd77f9c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:00.730147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76c356-f03a-45de-b999-2b4fd77f9c17 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:00.730653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76c356-f03a-45de-b999-2b4fd77f9c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:00.731177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76c356-f03a-45de-b999-2b4fd77f9c17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:00.743028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76c356-f03a-45de-b999-2b4fd77f9c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-dc76c356-f03a-45de-b999-2b4fd77f9c17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:54:00.749351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:00.750311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:54:00.751217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:00.753897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 436/854] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:54:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:00.762413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08e1d8-7480-471b-bbe4-55992cf16fe4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:00.828525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb58bd2-7243-41e4-8cba-34f06c3fd240 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:00.829043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb58bd2-7243-41e4-8cba-34f06c3fd240 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:00.829043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb58bd2-7243-41e4-8cba-34f06c3fd240 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:00.834306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:00.835086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 437/855] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:54:00.836349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176c3c4-6c07-4794-9f8b-7bafa926ff22 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:00.836707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176c3c4-6c07-4794-9f8b-7bafa926ff22 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:00.836985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176c3c4-6c07-4794-9f8b-7bafa926ff22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:00.861522 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8176c3c4-6c07-4794-9f8b-7bafa926ff22 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:00.862724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8176c3c4-6c07-4794-9f8b-7bafa926ff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:00.887024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08e1d8-7480-471b-bbe4-55992cf16fe4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:00.941551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f459c-35fd-40f0-bb77-5baa5396db35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:00.942160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f459c-35fd-40f0-bb77-5baa5396db35 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:00.942433 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f459c-35fd-40f0-bb77-5baa5396db35 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:00.942698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f459c-35fd-40f0-bb77-5baa5396db35 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:00.943475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f459c-35fd-40f0-bb77-5baa5396db35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:00.943777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f459c-35fd-40f0-bb77-5baa5396db35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:00.945141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f459c-35fd-40f0-bb77-5baa5396db35 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:00.948310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 438/856] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:00.959077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 419/857] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:54:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:00.967019 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176c3c4-6c07-4794-9f8b-7bafa926ff22 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:01.027454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:01.075248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176c3c4-6c07-4794-9f8b-7bafa926ff22 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:01.110930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c284ca-cc97-44d6-b5ef-0beb1fb27546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:01.111915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c284ca-cc97-44d6-b5ef-0beb1fb27546 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:01.112398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c284ca-cc97-44d6-b5ef-0beb1fb27546 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:01.112908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c284ca-cc97-44d6-b5ef-0beb1fb27546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:01.126399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c284ca-cc97-44d6-b5ef-0beb1fb27546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:01.128038 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c284ca-cc97-44d6-b5ef-0beb1fb27546 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-c6c284ca-cc97-44d6-b5ef-0beb1fb27546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:01.145032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:01.146066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:01.146995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:01.146995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:01.146995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:01.146995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 439/858] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:01.152108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f459c-35fd-40f0-bb77-5baa5396db35 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:01.222386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c284ca-cc97-44d6-b5ef-0beb1fb27546 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:01.272028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f459c-35fd-40f0-bb77-5baa5396db35 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:01.300073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:01.313190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fd1c4-22a9-4383-9dc3-71b1f16a7741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:01.314050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fd1c4-22a9-4383-9dc3-71b1f16a7741 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:01.314492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fd1c4-22a9-4383-9dc3-71b1f16a7741 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:01.314761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fd1c4-22a9-4383-9dc3-71b1f16a7741 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:01.315429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fd1c4-22a9-4383-9dc3-71b1f16a7741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:01.315729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fd1c4-22a9-4383-9dc3-71b1f16a7741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:01.317089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fd1c4-22a9-4383-9dc3-71b1f16a7741 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:01.329512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 420/859] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:01.330610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40813-8585-4345-878b-b80dae87c122 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:01.330888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40813-8585-4345-878b-b80dae87c122 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:01.331197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40813-8585-4345-878b-b80dae87c122 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:01.332512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 421/860] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:01.335786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40813-8585-4345-878b-b80dae87c122 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:01.351659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c284ca-cc97-44d6-b5ef-0beb1fb27546 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:01.366787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:01.391096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1bc6e-a7c7-4c7c-a316-27a5739e8ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:01.391749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1bc6e-a7c7-4c7c-a316-27a5739e8ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:01.392024 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1bc6e-a7c7-4c7c-a316-27a5739e8ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:01.392299 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1bc6e-a7c7-4c7c-a316-27a5739e8ec4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:01.392998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1bc6e-a7c7-4c7c-a316-27a5739e8ec4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:01.394496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1bc6e-a7c7-4c7c-a316-27a5739e8ec4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-eac1bc6e-a7c7-4c7c-a316-27a5739e8ec4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:01.404913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:01.405926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:01.406885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 440/861] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Sep 22 16:54:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:01.408177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd67bf-3e0d-4387-a490-d093781ff772 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:01.408177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd67bf-3e0d-4387-a490-d093781ff772 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:01.408177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd67bf-3e0d-4387-a490-d093781ff772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:01.426644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-80fd67bf-3e0d-4387-a490-d093781ff772 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:01.427868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-80fd67bf-3e0d-4387-a490-d093781ff772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:01.456602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40813-8585-4345-878b-b80dae87c122 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:01.491300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbf2f3-900f-4f45-a61b-73babcd268ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:01.491923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbf2f3-900f-4f45-a61b-73babcd268ae None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:01.492208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbf2f3-900f-4f45-a61b-73babcd268ae None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:01.492464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbf2f3-900f-4f45-a61b-73babcd268ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:01.500170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbf2f3-900f-4f45-a61b-73babcd268ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:01.501052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 441/862] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:01.501610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbf2f3-900f-4f45-a61b-73babcd268ae None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:01.502329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd67bf-3e0d-4387-a490-d093781ff772 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-adcbf2f3-900f-4f45-a61b-73babcd268ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:54:01.531021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:01.532076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:01.533037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:01.533037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:01.533037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:54:01.533037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:54:01.533037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:54:01.533037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:54:01.533037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:54:01.533037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:54:01.533037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:54:01.533037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:01.535114 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 422/863] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 22 16:54:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:01.539260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbf2f3-900f-4f45-a61b-73babcd268ae None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:01.601498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:01.621533 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd67bf-3e0d-4387-a490-d093781ff772 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:01.628990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:01.662635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ea299-b346-4e46-a0fc-41d0d4312aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:01.663279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ea299-b346-4e46-a0fc-41d0d4312aeb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:01.663575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ea299-b346-4e46-a0fc-41d0d4312aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:01.663817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ea299-b346-4e46-a0fc-41d0d4312aeb None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:01.664432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ea299-b346-4e46-a0fc-41d0d4312aeb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:01.664730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ea299-b346-4e46-a0fc-41d0d4312aeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:01.666073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ea299-b346-4e46-a0fc-41d0d4312aeb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:01.675319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 442/864] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 22 16:54:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:01.676909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ea299-b346-4e46-a0fc-41d0d4312aeb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:01.682577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbf2f3-900f-4f45-a61b-73babcd268ae None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:01.724065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b4785-ee2c-4c07-9eee-1c21469f5332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:01.724987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b4785-ee2c-4c07-9eee-1c21469f5332 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:01.724987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b4785-ee2c-4c07-9eee-1c21469f5332 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:01.725179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b4785-ee2c-4c07-9eee-1c21469f5332 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:01.725903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b4785-ee2c-4c07-9eee-1c21469f5332 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:01.726201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b4785-ee2c-4c07-9eee-1c21469f5332 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:01.727973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b4785-ee2c-4c07-9eee-1c21469f5332 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:01.756832 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 423/865] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 22 16:54:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:01.758843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b4785-ee2c-4c07-9eee-1c21469f5332 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:01.806211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ea299-b346-4e46-a0fc-41d0d4312aeb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:01.847937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeec43-76ab-4592-9464-358c335a98b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:01.849207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeec43-76ab-4592-9464-358c335a98b2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:01.849629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeec43-76ab-4592-9464-358c335a98b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:01.850058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeec43-76ab-4592-9464-358c335a98b2 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:01.850826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeec43-76ab-4592-9464-358c335a98b2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:01.851311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeec43-76ab-4592-9464-358c335a98b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:01.852798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeec43-76ab-4592-9464-358c335a98b2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:01.858258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 443/866] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 22 16:54:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:01.859272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeec43-76ab-4592-9464-358c335a98b2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:01.884539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b4785-ee2c-4c07-9eee-1c21469f5332 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:01.887291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:01.903395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:01.933182 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e07eb-04bf-4959-9a2b-6422f2533a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:01.934324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e07eb-04bf-4959-9a2b-6422f2533a1c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:01.934608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e07eb-04bf-4959-9a2b-6422f2533a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:01.935054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e07eb-04bf-4959-9a2b-6422f2533a1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:01.983165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeec43-76ab-4592-9464-358c335a98b2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:02.021531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99803c7b-91b6-454d-8ced-61aabf1d7f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:02.022154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99803c7b-91b6-454d-8ced-61aabf1d7f50 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:02.022435 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99803c7b-91b6-454d-8ced-61aabf1d7f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:02.022707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99803c7b-91b6-454d-8ced-61aabf1d7f50 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:02.023418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99803c7b-91b6-454d-8ced-61aabf1d7f50 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:02.024822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99803c7b-91b6-454d-8ced-61aabf1d7f50 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:02.025233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e07eb-04bf-4959-9a2b-6422f2533a1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:02.027599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e07eb-04bf-4959-9a2b-6422f2533a1c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:02.044538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 444/867] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:54:01 2022] POST /identity/v3/services => generated 251 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:02.155515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 424/868] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:54:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:54:02.161195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99803c7b-91b6-454d-8ced-61aabf1d7f50 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:02.270267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:02.285930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99803c7b-91b6-454d-8ced-61aabf1d7f50 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:02.323295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143c912-c0c8-4fbf-93c7-6c4abf813b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:02.323945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143c912-c0c8-4fbf-93c7-6c4abf813b03 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:02.324570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143c912-c0c8-4fbf-93c7-6c4abf813b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:02.324849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143c912-c0c8-4fbf-93c7-6c4abf813b03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:02.338743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143c912-c0c8-4fbf-93c7-6c4abf813b03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:02.340234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143c912-c0c8-4fbf-93c7-6c4abf813b03 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a143c912-c0c8-4fbf-93c7-6c4abf813b03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:02.355051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:02.356036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:02.356940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:02.356940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:02.356940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:02.356940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 445/869] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:02.361885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e07eb-04bf-4959-9a2b-6422f2533a1c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:02.364099 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:02.473382 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e07eb-04bf-4959-9a2b-6422f2533a1c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:02.509884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d9ab0-f0a5-438d-945d-d47d0c1db3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:02.510634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d9ab0-f0a5-438d-945d-d47d0c1db3aa None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:02.510913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d9ab0-f0a5-438d-945d-d47d0c1db3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:02.511280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d9ab0-f0a5-438d-945d-d47d0c1db3aa None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:02.511938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d9ab0-f0a5-438d-945d-d47d0c1db3aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:02.512238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d9ab0-f0a5-438d-945d-d47d0c1db3aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:02.513603 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d9ab0-f0a5-438d-945d-d47d0c1db3aa None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:02.526313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 425/870] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:02.531679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143c912-c0c8-4fbf-93c7-6c4abf813b03 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:02.637808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143c912-c0c8-4fbf-93c7-6c4abf813b03 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:02.674041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17345a4c-baee-43b4-bd34-afb88e91f57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:02.674686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345a4c-baee-43b4-bd34-afb88e91f57f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:02.674988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345a4c-baee-43b4-bd34-afb88e91f57f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:02.675269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345a4c-baee-43b4-bd34-afb88e91f57f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:02.682550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17345a4c-baee-43b4-bd34-afb88e91f57f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:02.684024 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17345a4c-baee-43b4-bd34-afb88e91f57f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-17345a4c-baee-43b4-bd34-afb88e91f57f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:54:02.695958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:02.696909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:02.698353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:54:02.698353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:54:02.698353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:54:02.698353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:54:02.698353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:54:02.698353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:54:02.698353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:02.698353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:02.698353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 446/871] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:02.703785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d9ab0-f0a5-438d-945d-d47d0c1db3aa None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:02.780593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:02.830056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d9ab0-f0a5-438d-945d-d47d0c1db3aa None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:02.847131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b712e-2317-4d8c-917a-f747d0f27a32 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:02.847446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b712e-2317-4d8c-917a-f747d0f27a32 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:02.847687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b712e-2317-4d8c-917a-f747d0f27a32 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:02.849276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 447/872] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:02.854699 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffe450-5b33-43a7-8f84-b8ae7e932f24 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:02.855017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffe450-5b33-43a7-8f84-b8ae7e932f24 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:02.855295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffe450-5b33-43a7-8f84-b8ae7e932f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:02.863277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:02.868587 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29176b-c59e-4248-9401-81cf60eb4242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:02.869206 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29176b-c59e-4248-9401-81cf60eb4242 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:02.869688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29176b-c59e-4248-9401-81cf60eb4242 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:02.870141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29176b-c59e-4248-9401-81cf60eb4242 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:02.872135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29176b-c59e-4248-9401-81cf60eb4242 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:02.872696 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29176b-c59e-4248-9401-81cf60eb4242 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:02.874179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29176b-c59e-4248-9401-81cf60eb4242 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:02.877607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f6ffe450-5b33-43a7-8f84-b8ae7e932f24 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:02.878895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f6ffe450-5b33-43a7-8f84-b8ae7e932f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:02.888579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 426/873] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:54:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:02.896203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29176b-c59e-4248-9401-81cf60eb4242 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:02.968898 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 448/874] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:03.023888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29176b-c59e-4248-9401-81cf60eb4242 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:03.050895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:03.066051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694d0f4-c076-4593-87a6-5f1892da4111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:03.066653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d0f4-c076-4593-87a6-5f1892da4111 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:03.066925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d0f4-c076-4593-87a6-5f1892da4111 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:03.067210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d0f4-c076-4593-87a6-5f1892da4111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:03.067935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694d0f4-c076-4593-87a6-5f1892da4111 None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=abbbe2b3f23b4c1caf8e20825cc2cfee) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:03.068234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694d0f4-c076-4593-87a6-5f1892da4111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:03.069756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694d0f4-c076-4593-87a6-5f1892da4111 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:03.076634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 427/875] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:54:02 2022] GET /identity/v3/role_assignments?user.id=abbbe2b3f23b4c1caf8e20825cc2cfee&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:03.240786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffe450-5b33-43a7-8f84-b8ae7e932f24 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:03.342403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffe450-5b33-43a7-8f84-b8ae7e932f24 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:03.373410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:03.378730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94fe1b-1843-4252-ab24-6da9f08bd892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:03.379404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94fe1b-1843-4252-ab24-6da9f08bd892 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:03.379735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94fe1b-1843-4252-ab24-6da9f08bd892 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:03.379945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94fe1b-1843-4252-ab24-6da9f08bd892 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:03.380519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94fe1b-1843-4252-ab24-6da9f08bd892 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:03.381931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94fe1b-1843-4252-ab24-6da9f08bd892 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-4a94fe1b-1843-4252-ab24-6da9f08bd892 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:03.391623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:03.392635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:03.393485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:03.397571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 449/876] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 22 16:54:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:03.398982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694d0f4-c076-4593-87a6-5f1892da4111 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:03.503362 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694d0f4-c076-4593-87a6-5f1892da4111 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:03.537434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ae964-37c0-4bcc-b4e2-5a4d53752d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:03.538035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ae964-37c0-4bcc-b4e2-5a4d53752d6a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:03.538311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ae964-37c0-4bcc-b4e2-5a4d53752d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:03.538631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ae964-37c0-4bcc-b4e2-5a4d53752d6a None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:03.539294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ae964-37c0-4bcc-b4e2-5a4d53752d6a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:03.539591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ae964-37c0-4bcc-b4e2-5a4d53752d6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:03.540984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ae964-37c0-4bcc-b4e2-5a4d53752d6a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:03.551073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 428/877] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 22 16:54:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:03.552531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94fe1b-1843-4252-ab24-6da9f08bd892 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:03.659270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94fe1b-1843-4252-ab24-6da9f08bd892 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:03.694613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af53e41-5c90-46c5-aadb-9cfd87584013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:03.695316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af53e41-5c90-46c5-aadb-9cfd87584013 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:03.695591 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af53e41-5c90-46c5-aadb-9cfd87584013 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:03.695858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af53e41-5c90-46c5-aadb-9cfd87584013 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:03.696473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af53e41-5c90-46c5-aadb-9cfd87584013 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:03.696767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af53e41-5c90-46c5-aadb-9cfd87584013 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:03.698145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af53e41-5c90-46c5-aadb-9cfd87584013 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:03.703174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 450/878] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 22 16:54:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:04.266296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3288956-ca5b-4ddd-855a-13759f7f35c7 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:04.267142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3288956-ca5b-4ddd-855a-13759f7f35c7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:04.267142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3288956-ca5b-4ddd-855a-13759f7f35c7 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:04.269102 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 429/879] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:54:04.275857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faf391-6aff-4d28-9231-84b6a1cf3e23 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:04.276439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faf391-6aff-4d28-9231-84b6a1cf3e23 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:04.276653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faf391-6aff-4d28-9231-84b6a1cf3e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:04.296473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-55faf391-6aff-4d28-9231-84b6a1cf3e23 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:04.297653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-55faf391-6aff-4d28-9231-84b6a1cf3e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:04.358604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:04.372043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 451/880] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:04.385503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d75df-a1d1-4fc3-af55-f79cb3189cab None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:04.385775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d75df-a1d1-4fc3-af55-f79cb3189cab None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:04.386050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d75df-a1d1-4fc3-af55-f79cb3189cab None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:04.387672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 430/881] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:04.392960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de47341-4770-41cb-a692-21c65bf61c31 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:04.393246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de47341-4770-41cb-a692-21c65bf61c31 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:04.393516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de47341-4770-41cb-a692-21c65bf61c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:04.412722 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3de47341-4770-41cb-a692-21c65bf61c31 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:04.413901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3de47341-4770-41cb-a692-21c65bf61c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:04.484883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 452/882] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:04.639758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d75df-a1d1-4fc3-af55-f79cb3189cab None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:04.693627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:04.749532 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d75df-a1d1-4fc3-af55-f79cb3189cab None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:04.764139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de47341-4770-41cb-a692-21c65bf61c31 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:04.799581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165351f-7ddc-4d83-8c9b-dc52f9db0a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:04.801394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165351f-7ddc-4d83-8c9b-dc52f9db0a87 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:04.801902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165351f-7ddc-4d83-8c9b-dc52f9db0a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:04.802383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165351f-7ddc-4d83-8c9b-dc52f9db0a87 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:04.803267 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165351f-7ddc-4d83-8c9b-dc52f9db0a87 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:04.804848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165351f-7ddc-4d83-8c9b-dc52f9db0a87 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:04.825640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 431/883] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:54:04 2022] POST /identity/v3/services => generated 269 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:04.884551 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de47341-4770-41cb-a692-21c65bf61c31 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:04.909964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:04.934829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1532c-a4b0-493d-ab5d-6f3846be5c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:04.935511 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1532c-a4b0-493d-ab5d-6f3846be5c66 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:04.935844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1532c-a4b0-493d-ab5d-6f3846be5c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:04.936120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1532c-a4b0-493d-ab5d-6f3846be5c66 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:04.936754 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1532c-a4b0-493d-ab5d-6f3846be5c66 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:04.938179 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1532c-a4b0-493d-ab5d-6f3846be5c66 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-dab1532c-a4b0-493d-ab5d-6f3846be5c66 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:04.948117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:04.949196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:04.950063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 453/884] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:54:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:04.955759 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165351f-7ddc-4d83-8c9b-dc52f9db0a87 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:05.070663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165351f-7ddc-4d83-8c9b-dc52f9db0a87 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:05.085134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:05.110774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65923db-188e-49af-b9ea-e3a6f5442e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:05.111407 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65923db-188e-49af-b9ea-e3a6f5442e85 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:05.111685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65923db-188e-49af-b9ea-e3a6f5442e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:05.111955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65923db-188e-49af-b9ea-e3a6f5442e85 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:05.112582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65923db-188e-49af-b9ea-e3a6f5442e85 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:05.112881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65923db-188e-49af-b9ea-e3a6f5442e85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:05.114242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65923db-188e-49af-b9ea-e3a6f5442e85 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:05.119348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 432/885] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:54:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:05.124716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1532c-a4b0-493d-ab5d-6f3846be5c66 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:05.232553 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1532c-a4b0-493d-ab5d-6f3846be5c66 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:05.269376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640c580-8d94-4530-bb33-e1d07f6b79c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:05.270040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640c580-8d94-4530-bb33-e1d07f6b79c2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:05.270322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640c580-8d94-4530-bb33-e1d07f6b79c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:05.270594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640c580-8d94-4530-bb33-e1d07f6b79c2 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:05.271253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640c580-8d94-4530-bb33-e1d07f6b79c2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:05.271553 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640c580-8d94-4530-bb33-e1d07f6b79c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:05.272909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640c580-8d94-4530-bb33-e1d07f6b79c2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:05.278229 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 454/886] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:54:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:05.284014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65923db-188e-49af-b9ea-e3a6f5442e85 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:05.394363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65923db-188e-49af-b9ea-e3a6f5442e85 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:05.430142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638693b0-5385-431e-a48a-4b5fd34db299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:05.430769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638693b0-5385-431e-a48a-4b5fd34db299 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:05.431079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638693b0-5385-431e-a48a-4b5fd34db299 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:05.431353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638693b0-5385-431e-a48a-4b5fd34db299 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:05.432626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638693b0-5385-431e-a48a-4b5fd34db299 None admin] RBAC: Adding query filter params (interface=public, service_id=133658b0267e40908cb669bb24bdfc3b, region_id=RegionOne) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:05.432935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638693b0-5385-431e-a48a-4b5fd34db299 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:05.434312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638693b0-5385-431e-a48a-4b5fd34db299 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:05.442420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 433/887] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 22 16:54:05 2022] GET /identity/v3/endpoints?service_id=133658b0267e40908cb669bb24bdfc3b&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:05.448703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640c580-8d94-4530-bb33-e1d07f6b79c2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:05.500285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:05.563392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640c580-8d94-4530-bb33-e1d07f6b79c2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:05.599216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c504d-e0f5-4a48-9daa-1fd312cd0db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:05.599841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c504d-e0f5-4a48-9daa-1fd312cd0db4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:05.600117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c504d-e0f5-4a48-9daa-1fd312cd0db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:05.600385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c504d-e0f5-4a48-9daa-1fd312cd0db4 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:05.600993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c504d-e0f5-4a48-9daa-1fd312cd0db4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:05.602366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c504d-e0f5-4a48-9daa-1fd312cd0db4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:05.609100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 455/888] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 22 16:54:05 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 16:54:05.637978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3aa67-4e84-4b16-987e-2e7cd963c352 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:05.638641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3aa67-4e84-4b16-987e-2e7cd963c352 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:05.639235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3aa67-4e84-4b16-987e-2e7cd963c352 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:05.641227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 434/889] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:05.647353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a19874-3f72-40a3-8820-5baa0f75f379 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:05.647928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a19874-3f72-40a3-8820-5baa0f75f379 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:05.648427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a19874-3f72-40a3-8820-5baa0f75f379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:05.668759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-45a19874-3f72-40a3-8820-5baa0f75f379 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:05.670303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-45a19874-3f72-40a3-8820-5baa0f75f379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:05.754416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 456/890] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:06.017864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65d54-2eb9-40c9-9daa-5ff717fb59cf None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:06.018063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65d54-2eb9-40c9-9daa-5ff717fb59cf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:06.018325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65d54-2eb9-40c9-9daa-5ff717fb59cf None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:06.020243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 435/891] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:06.026022 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54ed9-c9d8-4b2e-8feb-af57830e0895 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:06.026475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54ed9-c9d8-4b2e-8feb-af57830e0895 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:06.026920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54ed9-c9d8-4b2e-8feb-af57830e0895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:06.028980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a65d54-2eb9-40c9-9daa-5ff717fb59cf None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:06.052814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3be54ed9-c9d8-4b2e-8feb-af57830e0895 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:06.054454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3be54ed9-c9d8-4b2e-8feb-af57830e0895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:06.078287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:06.098438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:06.149155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 457/892] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:06.163610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a65d54-2eb9-40c9-9daa-5ff717fb59cf None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:06.200486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9ed67-9c44-4dfb-9642-213efce8c5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:06.201133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9ed67-9c44-4dfb-9642-213efce8c5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:06.201474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9ed67-9c44-4dfb-9642-213efce8c5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:06.201749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9ed67-9c44-4dfb-9642-213efce8c5d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:06.213194 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9ed67-9c44-4dfb-9642-213efce8c5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-33b9ed67-9c44-4dfb-9642-213efce8c5d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:54:06.218156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:06.219119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:06.219959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 436/893] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 22 16:54:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:06.225760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be54ed9-c9d8-4b2e-8feb-af57830e0895 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:06.330377 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be54ed9-c9d8-4b2e-8feb-af57830e0895 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:06.369529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff3b80-ea83-4f92-a473-5af5dea541a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:06.370142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff3b80-ea83-4f92-a473-5af5dea541a5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:06.370421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff3b80-ea83-4f92-a473-5af5dea541a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:06.370694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff3b80-ea83-4f92-a473-5af5dea541a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:06.371389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff3b80-ea83-4f92-a473-5af5dea541a5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:06.371692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff3b80-ea83-4f92-a473-5af5dea541a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:06.373111 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff3b80-ea83-4f92-a473-5af5dea541a5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:06.382098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 458/894] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 22 16:54:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:06.388119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9ed67-9c44-4dfb-9642-213efce8c5d2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:06.437147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff3b80-ea83-4f92-a473-5af5dea541a5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:06.471288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:06.512219 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9ed67-9c44-4dfb-9642-213efce8c5d2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:06.515351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:06.550324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e66d5-811f-4926-b91a-40a9d3ecc864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:06.551369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e66d5-811f-4926-b91a-40a9d3ecc864 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:06.551855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e66d5-811f-4926-b91a-40a9d3ecc864 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:06.552328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e66d5-811f-4926-b91a-40a9d3ecc864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:06.563791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e66d5-811f-4926-b91a-40a9d3ecc864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:06.564753 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff3b80-ea83-4f92-a473-5af5dea541a5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:06.565816 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e66d5-811f-4926-b91a-40a9d3ecc864 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-d15e66d5-811f-4926-b91a-40a9d3ecc864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:06.583599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:06.584582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:06.585484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:06.585484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:06.585484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:06.592482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 437/895] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:06.594360 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e66d5-811f-4926-b91a-40a9d3ecc864 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:06.610824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb921c44-2265-4fe1-8001-24c7f54a8042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:06.611538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb921c44-2265-4fe1-8001-24c7f54a8042 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:06.611838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb921c44-2265-4fe1-8001-24c7f54a8042 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:06.612109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb921c44-2265-4fe1-8001-24c7f54a8042 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:06.613028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb921c44-2265-4fe1-8001-24c7f54a8042 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:06.614890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb921c44-2265-4fe1-8001-24c7f54a8042 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:06.635756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 459/896] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:54:06 2022] POST /identity/v3/services => generated 241 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 16:54:06.725522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e66d5-811f-4926-b91a-40a9d3ecc864 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:06.775078 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2a643-200e-4e6f-bd97-46e4ef0db85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:06.776410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2a643-200e-4e6f-bd97-46e4ef0db85b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:06.776689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2a643-200e-4e6f-bd97-46e4ef0db85b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:06.776959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2a643-200e-4e6f-bd97-46e4ef0db85b None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:06.777621 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2a643-200e-4e6f-bd97-46e4ef0db85b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:06.777920 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2a643-200e-4e6f-bd97-46e4ef0db85b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:06.779382 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2a643-200e-4e6f-bd97-46e4ef0db85b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:06.794707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 438/897] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:06.801183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb921c44-2265-4fe1-8001-24c7f54a8042 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:06.903189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:06.916358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb921c44-2265-4fe1-8001-24c7f54a8042 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:06.953187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad4be9-6351-4b24-ba17-6ef2bbcd5372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:06.953818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad4be9-6351-4b24-ba17-6ef2bbcd5372 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:06.954092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad4be9-6351-4b24-ba17-6ef2bbcd5372 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:06.954358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad4be9-6351-4b24-ba17-6ef2bbcd5372 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:06.963490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad4be9-6351-4b24-ba17-6ef2bbcd5372 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:06.964913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad4be9-6351-4b24-ba17-6ef2bbcd5372 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-00ad4be9-6351-4b24-ba17-6ef2bbcd5372 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:54:06.976447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:06.977441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:06.978795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:06.978795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:06.978795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:54:06.978795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:54:06.978795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:54:06.978795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:54:06.978795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:54:06.978795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:54:06.978795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:54:06.978795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:06.978795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 460/898] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 22 16:54:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:06.983341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2a643-200e-4e6f-bd97-46e4ef0db85b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:06.992518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:07.100668 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2a643-200e-4e6f-bd97-46e4ef0db85b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:07.139866 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318e7f9-6ff5-4bdd-adbb-5e271846c7ed None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:07.140148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318e7f9-6ff5-4bdd-adbb-5e271846c7ed None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:07.140422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318e7f9-6ff5-4bdd-adbb-5e271846c7ed None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:07.141997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 461/899] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:54:07.145510 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945be7d2-7eee-4c94-a2f7-87db24c81d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:07.146337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be7d2-7eee-4c94-a2f7-87db24c81d61 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:07.146812 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be7d2-7eee-4c94-a2f7-87db24c81d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:07.146919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afb07f-98a5-43ac-beda-7d00cd8c6105 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:07.147248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afb07f-98a5-43ac-beda-7d00cd8c6105 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:07.147521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afb07f-98a5-43ac-beda-7d00cd8c6105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:07.147764 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be7d2-7eee-4c94-a2f7-87db24c81d61 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:07.148723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945be7d2-7eee-4c94-a2f7-87db24c81d61 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:07.149213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945be7d2-7eee-4c94-a2f7-87db24c81d61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:07.151200 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945be7d2-7eee-4c94-a2f7-87db24c81d61 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:07.165688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a9afb07f-98a5-43ac-beda-7d00cd8c6105 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:07.167895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a9afb07f-98a5-43ac-beda-7d00cd8c6105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:07.172151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 439/900] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 22 16:54:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:07.180688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945be7d2-7eee-4c94-a2f7-87db24c81d61 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:07.261376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:07.263223 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 462/901] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:07.302027 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945be7d2-7eee-4c94-a2f7-87db24c81d61 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:07.337564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6194f30-f305-4322-bba5-758b8f2f25ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:07.338215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6194f30-f305-4322-bba5-758b8f2f25ab None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:07.338492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6194f30-f305-4322-bba5-758b8f2f25ab None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:07.338764 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6194f30-f305-4322-bba5-758b8f2f25ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:07.415666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6194f30-f305-4322-bba5-758b8f2f25ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:07.418515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6194f30-f305-4322-bba5-758b8f2f25ab None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:07.487610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:07.532359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afb07f-98a5-43ac-beda-7d00cd8c6105 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:07.536143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 440/902] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:54:07 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:54:07.543988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6194f30-f305-4322-bba5-758b8f2f25ab None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:07.633221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:07.655832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afb07f-98a5-43ac-beda-7d00cd8c6105 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:07.679202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6194f30-f305-4322-bba5-758b8f2f25ab None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:07.694620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be121999-0a33-4224-8c3a-ae1e718a8590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:07.695320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be121999-0a33-4224-8c3a-ae1e718a8590 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:07.695598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be121999-0a33-4224-8c3a-ae1e718a8590 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:07.695868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be121999-0a33-4224-8c3a-ae1e718a8590 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:07.696579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be121999-0a33-4224-8c3a-ae1e718a8590 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:07.698737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be121999-0a33-4224-8c3a-ae1e718a8590 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-be121999-0a33-4224-8c3a-ae1e718a8590 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:07.709106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:07.710169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 16:54:07.711118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:07.715370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 463/903] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Sep 22 16:54:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:07.717337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be121999-0a33-4224-8c3a-ae1e718a8590 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:07.732969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbca23-d751-4230-bf77-72466dc5d8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:07.734212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbca23-d751-4230-bf77-72466dc5d8fb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:07.735145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbca23-d751-4230-bf77-72466dc5d8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:07.735391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbca23-d751-4230-bf77-72466dc5d8fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:07.751136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbca23-d751-4230-bf77-72466dc5d8fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:07.753005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbca23-d751-4230-bf77-72466dc5d8fb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-fedbca23-d751-4230-bf77-72466dc5d8fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:07.774822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:07.776615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:07.777506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:07.777506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:07.777506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:07.782928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 441/904] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:07.784128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbca23-d751-4230-bf77-72466dc5d8fb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:07.810478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:07.855239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be121999-0a33-4224-8c3a-ae1e718a8590 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:07.897553 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52af6ce-da3b-4d13-9e75-1d250f30eb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:07.898167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52af6ce-da3b-4d13-9e75-1d250f30eb79 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:07.898456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52af6ce-da3b-4d13-9e75-1d250f30eb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:07.898737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52af6ce-da3b-4d13-9e75-1d250f30eb79 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:07.899385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52af6ce-da3b-4d13-9e75-1d250f30eb79 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:07.899687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52af6ce-da3b-4d13-9e75-1d250f30eb79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:07.901070 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52af6ce-da3b-4d13-9e75-1d250f30eb79 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:07.905189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbca23-d751-4230-bf77-72466dc5d8fb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:07.905958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 464/905] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 22 16:54:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:07.906990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86866d-a5be-405a-ac7c-b597768ee7e8 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:07.907281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86866d-a5be-405a-ac7c-b597768ee7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:07.907556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86866d-a5be-405a-ac7c-b597768ee7e8 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:07.912828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 465/906] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:07.912828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86866d-a5be-405a-ac7c-b597768ee7e8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:07.924822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:07.949787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe5f6c-b959-4cf3-9d2b-d7059b5ffd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:07.950601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe5f6c-b959-4cf3-9d2b-d7059b5ffd5f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:07.950782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe5f6c-b959-4cf3-9d2b-d7059b5ffd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:07.951063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe5f6c-b959-4cf3-9d2b-d7059b5ffd5f None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:07.951693 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe5f6c-b959-4cf3-9d2b-d7059b5ffd5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:07.951996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe5f6c-b959-4cf3-9d2b-d7059b5ffd5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:07.953346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe5f6c-b959-4cf3-9d2b-d7059b5ffd5f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:07.971266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 442/907] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:07.971266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e452f8-6dbe-40c6-8845-24e453e36f60 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:07.971266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e452f8-6dbe-40c6-8845-24e453e36f60 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:07.971266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e452f8-6dbe-40c6-8845-24e453e36f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:07.987553 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a8e452f8-6dbe-40c6-8845-24e453e36f60 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:07.988729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a8e452f8-6dbe-40c6-8845-24e453e36f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:08.033614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86866d-a5be-405a-ac7c-b597768ee7e8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:08.061389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 443/908] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:08.062306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e452f8-6dbe-40c6-8845-24e453e36f60 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:08.071008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:08.082792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755957e-bb3b-447b-a5c5-e34f602a5dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:08.084247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755957e-bb3b-447b-a5c5-e34f602a5dba None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:08.085417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755957e-bb3b-447b-a5c5-e34f602a5dba None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:08.085697 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755957e-bb3b-447b-a5c5-e34f602a5dba None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:08.086333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755957e-bb3b-447b-a5c5-e34f602a5dba None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:08.086634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755957e-bb3b-447b-a5c5-e34f602a5dba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:08.088211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755957e-bb3b-447b-a5c5-e34f602a5dba None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:08.093983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 466/909] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 22 16:54:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:08.099176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755957e-bb3b-447b-a5c5-e34f602a5dba None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:08.186418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:08.195399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e452f8-6dbe-40c6-8845-24e453e36f60 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:08.225969 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755957e-bb3b-447b-a5c5-e34f602a5dba None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:08.248189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a12d87-1f01-4dbe-915e-42ce3b037b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:08.249439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a12d87-1f01-4dbe-915e-42ce3b037b0b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:08.249862 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a12d87-1f01-4dbe-915e-42ce3b037b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:08.250298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a12d87-1f01-4dbe-915e-42ce3b037b0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:08.260446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a12d87-1f01-4dbe-915e-42ce3b037b0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:08.262634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a12d87-1f01-4dbe-915e-42ce3b037b0b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:08.269190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cad30-97c5-46e3-aeb6-5ac59847565c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:08.269964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cad30-97c5-46e3-aeb6-5ac59847565c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:08.270346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cad30-97c5-46e3-aeb6-5ac59847565c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:08.270871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cad30-97c5-46e3-aeb6-5ac59847565c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:08.271860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cad30-97c5-46e3-aeb6-5ac59847565c None admin] RBAC: Adding query filter params (interface=public, service_id=16a9c351a4364f12a88dfbd6eba3e30f, region_id=RegionOne) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:08.272319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cad30-97c5-46e3-aeb6-5ac59847565c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:08.273761 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cad30-97c5-46e3-aeb6-5ac59847565c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-97a12d87-1f01-4dbe-915e-42ce3b037b0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:54:08.278154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:08.279224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:08.280400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:54:08.280400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:54:08.280400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:54:08.280400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:54:08.280400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:54:08.280400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:54:08.280400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:08.280400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:08.282096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 467/910] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 22 16:54:08 2022] GET /identity/v3/endpoints?service_id=16a9c351a4364f12a88dfbd6eba3e30f&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:08.282425 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 444/911] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:08.287353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cad30-97c5-46e3-aeb6-5ac59847565c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:08.293019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a12d87-1f01-4dbe-915e-42ce3b037b0b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:08.372214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:08.405133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cad30-97c5-46e3-aeb6-5ac59847565c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:08.409320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a12d87-1f01-4dbe-915e-42ce3b037b0b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:08.441696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4decde-77c6-4c9a-bce2-c120a4bb0f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:08.442303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4decde-77c6-4c9a-bce2-c120a4bb0f43 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:08.442587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4decde-77c6-4c9a-bce2-c120a4bb0f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:08.442872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4decde-77c6-4c9a-bce2-c120a4bb0f43 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:08.443499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4decde-77c6-4c9a-bce2-c120a4bb0f43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:08.444337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1df76-783f-4170-8e5f-cfc2388d361a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:08.444853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4decde-77c6-4c9a-bce2-c120a4bb0f43 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:08.444949 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1df76-783f-4170-8e5f-cfc2388d361a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:08.445195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1df76-783f-4170-8e5f-cfc2388d361a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:08.445458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1df76-783f-4170-8e5f-cfc2388d361a None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:08.446129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1df76-783f-4170-8e5f-cfc2388d361a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:08.446429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1df76-783f-4170-8e5f-cfc2388d361a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:08.447085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:08.448429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1df76-783f-4170-8e5f-cfc2388d361a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:08.464634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 468/912] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 22 16:54:08 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 16:54:08.465992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4decde-77c6-4c9a-bce2-c120a4bb0f43 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:08.470110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 445/913] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:54:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:08.477390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1df76-783f-4170-8e5f-cfc2388d361a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:08.606660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4decde-77c6-4c9a-bce2-c120a4bb0f43 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:08.607666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1df76-783f-4170-8e5f-cfc2388d361a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:08.647516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:08.654978 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c11a0-ab68-4004-a615-a258fac31ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:08.656976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c11a0-ab68-4004-a615-a258fac31ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:08.657262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c11a0-ab68-4004-a615-a258fac31ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:08.657539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c11a0-ab68-4004-a615-a258fac31ab2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:08.658145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c11a0-ab68-4004-a615-a258fac31ab2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:08.659536 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c11a0-ab68-4004-a615-a258fac31ab2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:08.661796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af6e2f-f5e1-4d1b-bbf3-493f1173bfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:08.663021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af6e2f-f5e1-4d1b-bbf3-493f1173bfaa None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:08.663724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af6e2f-f5e1-4d1b-bbf3-493f1173bfaa None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:08.664171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af6e2f-f5e1-4d1b-bbf3-493f1173bfaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:08.664903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af6e2f-f5e1-4d1b-bbf3-493f1173bfaa None admin] RBAC: Adding query filter params (role.id=34a4f36a154c4c03942a6a78e370fb70, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=abbbe2b3f23b4c1caf8e20825cc2cfee) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:08.665211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af6e2f-f5e1-4d1b-bbf3-493f1173bfaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:08.667209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af6e2f-f5e1-4d1b-bbf3-493f1173bfaa None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-9e4c11a0-ab68-4004-a615-a258fac31ab2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:08.669475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:08.670419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:08.671373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 469/914] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:54:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:08.677157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c11a0-ab68-4004-a615-a258fac31ab2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:08.679171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 446/915] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:54:08 2022] GET /identity/v3/role_assignments?user.id=abbbe2b3f23b4c1caf8e20825cc2cfee&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=34a4f36a154c4c03942a6a78e370fb70 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:08.753302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:08.800004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c11a0-ab68-4004-a615-a258fac31ab2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:08.840338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499412a-6a08-4f55-8966-f08df5277d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:08.840974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499412a-6a08-4f55-8966-f08df5277d9a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:08.841253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499412a-6a08-4f55-8966-f08df5277d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:08.841526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499412a-6a08-4f55-8966-f08df5277d9a None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:08.842177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499412a-6a08-4f55-8966-f08df5277d9a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:08.842481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499412a-6a08-4f55-8966-f08df5277d9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:08.843907 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499412a-6a08-4f55-8966-f08df5277d9a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:08.849790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 470/916] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:54:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:08.855360 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af6e2f-f5e1-4d1b-bbf3-493f1173bfaa None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:08.977086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af6e2f-f5e1-4d1b-bbf3-493f1173bfaa None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:09.014027 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60771cd-c8b5-4c23-8822-bd200ea69a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:09.014555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60771cd-c8b5-4c23-8822-bd200ea69a60 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:09.014829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60771cd-c8b5-4c23-8822-bd200ea69a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:09.015128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60771cd-c8b5-4c23-8822-bd200ea69a60 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:09.015750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60771cd-c8b5-4c23-8822-bd200ea69a60 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:09.016047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60771cd-c8b5-4c23-8822-bd200ea69a60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:09.017458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60771cd-c8b5-4c23-8822-bd200ea69a60 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:09.022639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498d33a-38a7-45d0-a390-d35b4592264a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:09.022928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498d33a-38a7-45d0-a390-d35b4592264a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:09.023248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498d33a-38a7-45d0-a390-d35b4592264a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:09.024784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 471/918] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:09.027129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 447/918] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 22 16:54:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:09.028959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498d33a-38a7-45d0-a390-d35b4592264a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:09.031935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb13b0-0327-4f70-ae2b-06ae145c8821 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:09.032410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb13b0-0327-4f70-ae2b-06ae145c8821 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:09.032928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb13b0-0327-4f70-ae2b-06ae145c8821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:09.052229 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4feb13b0-0327-4f70-ae2b-06ae145c8821 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:09.053745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4feb13b0-0327-4f70-ae2b-06ae145c8821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:09.108262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:09.127379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 448/919] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:09.154211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498d33a-38a7-45d0-a390-d35b4592264a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:09.182734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:09.191861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e734876-b27d-4276-ac87-12603514e83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:09.192703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e734876-b27d-4276-ac87-12603514e83b None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:09.193130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e734876-b27d-4276-ac87-12603514e83b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:09.193552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e734876-b27d-4276-ac87-12603514e83b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:09.194369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e734876-b27d-4276-ac87-12603514e83b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:09.195920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e734876-b27d-4276-ac87-12603514e83b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:09.219863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 472/920] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 16:54:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:09.422330 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb13b0-0327-4f70-ae2b-06ae145c8821 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:09.540116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb13b0-0327-4f70-ae2b-06ae145c8821 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:09.592383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c28a1-4415-4fa7-85b5-065a39636b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:09.593024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c28a1-4415-4fa7-85b5-065a39636b2f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:09.593301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c28a1-4415-4fa7-85b5-065a39636b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:09.593686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c28a1-4415-4fa7-85b5-065a39636b2f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:09.594716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c28a1-4415-4fa7-85b5-065a39636b2f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:09.596239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c28a1-4415-4fa7-85b5-065a39636b2f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-714c28a1-4415-4fa7-85b5-065a39636b2f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:09.605493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:09.606512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:09.607392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 449/921] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 22 16:54:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:09.614691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e734876-b27d-4276-ac87-12603514e83b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:09.729030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e734876-b27d-4276-ac87-12603514e83b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:09.770450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77982fe6-12c3-401c-8d9c-692b4d8bb8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:09.771373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77982fe6-12c3-401c-8d9c-692b4d8bb8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:09.771857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77982fe6-12c3-401c-8d9c-692b4d8bb8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:09.772372 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77982fe6-12c3-401c-8d9c-692b4d8bb8c9 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:09.773202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77982fe6-12c3-401c-8d9c-692b4d8bb8c9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:09.773700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77982fe6-12c3-401c-8d9c-692b4d8bb8c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:09.775819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77982fe6-12c3-401c-8d9c-692b4d8bb8c9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:09.781424 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 473/922] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 22 16:54:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:09.791016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c28a1-4415-4fa7-85b5-065a39636b2f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:09.845096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:09.901591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c28a1-4415-4fa7-85b5-065a39636b2f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:09.937108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce5932-1db8-4105-a939-70ef024e2d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:09.937720 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce5932-1db8-4105-a939-70ef024e2d95 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:09.937993 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce5932-1db8-4105-a939-70ef024e2d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:09.938271 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce5932-1db8-4105-a939-70ef024e2d95 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:09.938903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce5932-1db8-4105-a939-70ef024e2d95 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:09.939238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce5932-1db8-4105-a939-70ef024e2d95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:09.940713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce5932-1db8-4105-a939-70ef024e2d95 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:09.950298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 450/923] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 22 16:54:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:09.955871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77982fe6-12c3-401c-8d9c-692b4d8bb8c9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:10.007230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:10.068325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77982fe6-12c3-401c-8d9c-692b4d8bb8c9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:10.104260 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e3cda-db01-457e-91aa-02c6873088db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:10.105244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e3cda-db01-457e-91aa-02c6873088db None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:10.105723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e3cda-db01-457e-91aa-02c6873088db None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:10.106194 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e3cda-db01-457e-91aa-02c6873088db None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:10.107123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e3cda-db01-457e-91aa-02c6873088db None admin] RBAC: Adding query filter params (interface=public, service_id=8f7eb0cc6fd24a668b485d996afe8f7b, region_id=RegionOne) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:10.107633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e3cda-db01-457e-91aa-02c6873088db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:10.109195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e3cda-db01-457e-91aa-02c6873088db None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:10.115087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 474/924] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 22 16:54:09 2022] GET /identity/v3/endpoints?service_id=8f7eb0cc6fd24a668b485d996afe8f7b&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:10.123718 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce5932-1db8-4105-a939-70ef024e2d95 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:10.228874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce5932-1db8-4105-a939-70ef024e2d95 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:10.239290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:10.267436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e286526-a4f2-4f34-9452-7dc219be1fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:10.267952 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e286526-a4f2-4f34-9452-7dc219be1fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:10.268231 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e286526-a4f2-4f34-9452-7dc219be1fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:10.268504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e286526-a4f2-4f34-9452-7dc219be1fb4 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:10.269112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e286526-a4f2-4f34-9452-7dc219be1fb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:10.270480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e286526-a4f2-4f34-9452-7dc219be1fb4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:10.277090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 451/925] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 22 16:54:10 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 16:54:10.941482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c29a4-e0da-42d5-b2cf-258b2d0a1baf None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:10.942014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c29a4-e0da-42d5-b2cf-258b2d0a1baf None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:10.942092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c29a4-e0da-42d5-b2cf-258b2d0a1baf None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:10.943763 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 475/926] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:10.951320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2038fc-0fdf-4a06-86e9-df4e3554108a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:10.951818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2038fc-0fdf-4a06-86e9-df4e3554108a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:10.951894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2038fc-0fdf-4a06-86e9-df4e3554108a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:10.972811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9b2038fc-0fdf-4a06-86e9-df4e3554108a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:10.974025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9b2038fc-0fdf-4a06-86e9-df4e3554108a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:11.063345 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 452/927] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:11.149437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2d456-180a-4cbf-b305-6a070e8dfc9e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:11.149896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2d456-180a-4cbf-b305-6a070e8dfc9e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:11.150402 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2d456-180a-4cbf-b305-6a070e8dfc9e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:11.152425 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 476/928] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:11.159370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b26d0-27d7-42f0-a926-ad398053cd9d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:11.159784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b26d0-27d7-42f0-a926-ad398053cd9d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:11.160125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b26d0-27d7-42f0-a926-ad398053cd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:11.181324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-133b26d0-27d7-42f0-a926-ad398053cd9d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:11.183207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-133b26d0-27d7-42f0-a926-ad398053cd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:11.259640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 453/929] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:11.369490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2d456-180a-4cbf-b305-6a070e8dfc9e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:11.493841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2d456-180a-4cbf-b305-6a070e8dfc9e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:11.512626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:11.541563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84ab81-6958-4c25-acff-f633ae163648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:11.541920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84ab81-6958-4c25-acff-f633ae163648 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:11.542259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84ab81-6958-4c25-acff-f633ae163648 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:11.542379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b26d0-27d7-42f0-a926-ad398053cd9d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:11.542544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84ab81-6958-4c25-acff-f633ae163648 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:11.543152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84ab81-6958-4c25-acff-f633ae163648 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:11.544524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84ab81-6958-4c25-acff-f633ae163648 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-6a84ab81-6958-4c25-acff-f633ae163648 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:11.556308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:11.558587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 16:54:11.560301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:11.562519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 477/930] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Sep 22 16:54:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:11.565440 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84ab81-6958-4c25-acff-f633ae163648 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:11.575572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:11.654346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b26d0-27d7-42f0-a926-ad398053cd9d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:11.675501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84ab81-6958-4c25-acff-f633ae163648 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:11.692224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ce78d-c5b3-4c46-bc13-4d6aea7b1325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:11.692848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce78d-c5b3-4c46-bc13-4d6aea7b1325 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:11.693153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce78d-c5b3-4c46-bc13-4d6aea7b1325 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:11.693382 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce78d-c5b3-4c46-bc13-4d6aea7b1325 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:11.705003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ce78d-c5b3-4c46-bc13-4d6aea7b1325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-1d4ce78d-c5b3-4c46-bc13-4d6aea7b1325 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:54:11.710421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:11.711652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:11.712490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:11.713575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b114fca-c464-46cc-9f2a-2cc2a7d053a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:11.713575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b114fca-c464-46cc-9f2a-2cc2a7d053a4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:11.713575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b114fca-c464-46cc-9f2a-2cc2a7d053a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:11.713575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b114fca-c464-46cc-9f2a-2cc2a7d053a4 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:11.713849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b114fca-c464-46cc-9f2a-2cc2a7d053a4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:11.714109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b114fca-c464-46cc-9f2a-2cc2a7d053a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:11.715427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b114fca-c464-46cc-9f2a-2cc2a7d053a4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:11.716921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 454/931] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 22 16:54:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:11.718128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5619a14-b904-4445-85b1-6ff684fc42b7 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:11.718411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5619a14-b904-4445-85b1-6ff684fc42b7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:11.718700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5619a14-b904-4445-85b1-6ff684fc42b7 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:11.725334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 455/932] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:11.725479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 478/933] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 22 16:54:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:11.725559 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5619a14-b904-4445-85b1-6ff684fc42b7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:11.727629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b114fca-c464-46cc-9f2a-2cc2a7d053a4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:11.776561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:11.833563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:11.843704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5619a14-b904-4445-85b1-6ff684fc42b7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:11.844372 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b114fca-c464-46cc-9f2a-2cc2a7d053a4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:11.883257 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba9f74-d872-4c09-ae24-7a74eefc4ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:11.883855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba9f74-d872-4c09-ae24-7a74eefc4ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:11.884133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba9f74-d872-4c09-ae24-7a74eefc4ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:11.884499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba9f74-d872-4c09-ae24-7a74eefc4ed6 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:11.884761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef34711-f5ff-48c9-bf63-b6851aba00ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:11.885065 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba9f74-d872-4c09-ae24-7a74eefc4ed6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:11.885321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba9f74-d872-4c09-ae24-7a74eefc4ed6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:11.885428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef34711-f5ff-48c9-bf63-b6851aba00ee None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:11.885632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef34711-f5ff-48c9-bf63-b6851aba00ee None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:11.885951 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef34711-f5ff-48c9-bf63-b6851aba00ee None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:11.886605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef34711-f5ff-48c9-bf63-b6851aba00ee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:11.886706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba9f74-d872-4c09-ae24-7a74eefc4ed6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:11.886902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef34711-f5ff-48c9-bf63-b6851aba00ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:11.888268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef34711-f5ff-48c9-bf63-b6851aba00ee None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:11.897157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 479/934] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 22 16:54:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:11.897157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226a1be-725d-4dbf-a9eb-af99fe36803c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:11.897157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226a1be-725d-4dbf-a9eb-af99fe36803c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:11.897157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226a1be-725d-4dbf-a9eb-af99fe36803c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:11.900871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 456/935] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 22 16:54:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:11.901682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef34711-f5ff-48c9-bf63-b6851aba00ee None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:11.917222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b226a1be-725d-4dbf-a9eb-af99fe36803c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:11.918333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b226a1be-725d-4dbf-a9eb-af99fe36803c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:11.998504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 480/936] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:11.999555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226a1be-725d-4dbf-a9eb-af99fe36803c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:12.037304 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef34711-f5ff-48c9-bf63-b6851aba00ee None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:12.054928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:12.074146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1886e-b516-4e8c-9e78-96eb49e9de6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:12.074854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1886e-b516-4e8c-9e78-96eb49e9de6b None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:12.075221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1886e-b516-4e8c-9e78-96eb49e9de6b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:12.075590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1886e-b516-4e8c-9e78-96eb49e9de6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:12.076329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1886e-b516-4e8c-9e78-96eb49e9de6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:12.077790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1886e-b516-4e8c-9e78-96eb49e9de6b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:12.106977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 457/937] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 16:54:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:12.154915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226a1be-725d-4dbf-a9eb-af99fe36803c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:12.203632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707b44d-9bfd-411b-8659-94f0577e195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:12.204282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707b44d-9bfd-411b-8659-94f0577e195e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:12.204564 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707b44d-9bfd-411b-8659-94f0577e195e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:12.204837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707b44d-9bfd-411b-8659-94f0577e195e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:12.217721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707b44d-9bfd-411b-8659-94f0577e195e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:12.219220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707b44d-9bfd-411b-8659-94f0577e195e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-5707b44d-9bfd-411b-8659-94f0577e195e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:12.237342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:12.238362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:12.239304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:12.239304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:12.239304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:12.239304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 481/938] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:12.244859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1886e-b516-4e8c-9e78-96eb49e9de6b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:12.261259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:12.321164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707b44d-9bfd-411b-8659-94f0577e195e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:12.370912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1886e-b516-4e8c-9e78-96eb49e9de6b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:12.409308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e9ba-9021-43a4-914e-ef26c844ab3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:12.410058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e9ba-9021-43a4-914e-ef26c844ab3d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:12.410347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e9ba-9021-43a4-914e-ef26c844ab3d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:12.410634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e9ba-9021-43a4-914e-ef26c844ab3d None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:12.411459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09e9ba-9021-43a4-914e-ef26c844ab3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:12.411790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09e9ba-9021-43a4-914e-ef26c844ab3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:12.413981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09e9ba-9021-43a4-914e-ef26c844ab3d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:12.435050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 458/939] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:12.439730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e9ba-9021-43a4-914e-ef26c844ab3d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:12.449084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:12.474153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707b44d-9bfd-411b-8659-94f0577e195e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:12.522141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa29b1-16a6-411a-ac9d-3804640ccc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:12.522788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa29b1-16a6-411a-ac9d-3804640ccc45 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:12.523099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa29b1-16a6-411a-ac9d-3804640ccc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:12.523376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa29b1-16a6-411a-ac9d-3804640ccc45 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:12.524310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa29b1-16a6-411a-ac9d-3804640ccc45 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:12.526452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa29b1-16a6-411a-ac9d-3804640ccc45 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:12.529377 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: WARNING keystone.server.flask.common [None req-d5aa29b1-16a6-411a-ac9d-3804640ccc45 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 22 16:54:12.533907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:12.558698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.sql.core [None req-d5aa29b1-16a6-411a-ac9d-3804640ccc45 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 16:54:12.558698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 16:54:12.558698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [parameters: {'id': '9642257d613b48ea8a9ba69be78c74c1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 16:54:12.558698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81561) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 16:54:12.574972 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e9ba-9021-43a4-914e-ef26c844ab3d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-d5aa29b1-16a6-411a-ac9d-3804640ccc45 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9642257d613b48ea8a9ba69be78c74c1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9642257d613b48ea8a9ba69be78c74c1. Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application conn.query(q) Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 16:54:12.575428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result.read() Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 22 16:54:12.576231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application next(self.gen) Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application yield resource Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application next(self.gen) Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise self.value Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application session.commit() Sep 22 16:54:12.577187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.session.flush() Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application compat.raise_( Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 16:54:12.578117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 16:54:12.579099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application conn.query(q) Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result.read() Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 16:54:12.579908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [parameters: {'id': '9642257d613b48ea8a9ba69be78c74c1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:12.583656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:12.585679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9642257d613b48ea8a9ba69be78c74c1. Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 482/940] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Sep 22 16:54:12 2022] POST /identity/v3/projects => generated 262 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d028102-fd7b-4871-88e1-079d8cf9ce43 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:12.588248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d028102-fd7b-4871-88e1-079d8cf9ce43 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:12.591000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d028102-fd7b-4871-88e1-079d8cf9ce43 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:12.591000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 483/941] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:12.591000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d028102-fd7b-4871-88e1-079d8cf9ce43 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:12.626667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb14516-5fce-42f3-b380-9922b5773c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:12.627342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14516-5fce-42f3-b380-9922b5773c6b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:12.627610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14516-5fce-42f3-b380-9922b5773c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:12.627875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14516-5fce-42f3-b380-9922b5773c6b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:12.637649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb14516-5fce-42f3-b380-9922b5773c6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:12.639126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb14516-5fce-42f3-b380-9922b5773c6b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-1bb14516-5fce-42f3-b380-9922b5773c6b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:54:12.652013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:12.653215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 459/942] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 22 16:54:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf2c92-ad7b-492d-a2e7-7c625ca3971e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:12.659032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf2c92-ad7b-492d-a2e7-7c625ca3971e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:12.659851 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf2c92-ad7b-492d-a2e7-7c625ca3971e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:12.677832 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-eecf2c92-ad7b-492d-a2e7-7c625ca3971e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:12.679184 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-eecf2c92-ad7b-492d-a2e7-7c625ca3971e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:12.704924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d028102-fd7b-4871-88e1-079d8cf9ce43 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:12.740950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fafa26-6c07-4e3b-8f3f-bf6bf1150bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:12.741828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fafa26-6c07-4e3b-8f3f-bf6bf1150bdb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:12.742296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fafa26-6c07-4e3b-8f3f-bf6bf1150bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:12.742703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fafa26-6c07-4e3b-8f3f-bf6bf1150bdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:12.752799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fafa26-6c07-4e3b-8f3f-bf6bf1150bdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:12.756583 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fafa26-6c07-4e3b-8f3f-bf6bf1150bdb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:12.763913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:12.771144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 460/943] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:12.771144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf2c92-ad7b-492d-a2e7-7c625ca3971e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b2fafa26-6c07-4e3b-8f3f-bf6bf1150bdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:54:12.773308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:12.774368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:12.776016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:54:12.776016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:54:12.776016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:54:12.776016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:54:12.776016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:54:12.776016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:54:12.776016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:12.776016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:12.778868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 484/944] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:12.784055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fafa26-6c07-4e3b-8f3f-bf6bf1150bdb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:12.860993 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:12.902618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fafa26-6c07-4e3b-8f3f-bf6bf1150bdb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:12.928069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf2c92-ad7b-492d-a2e7-7c625ca3971e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:12.951563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb225e36-b522-44d3-96dc-3981a41ef566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:12.952324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb225e36-b522-44d3-96dc-3981a41ef566 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:12.952709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb225e36-b522-44d3-96dc-3981a41ef566 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:12.953075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb225e36-b522-44d3-96dc-3981a41ef566 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:12.953860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb225e36-b522-44d3-96dc-3981a41ef566 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:12.954263 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb225e36-b522-44d3-96dc-3981a41ef566 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:12.955818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb225e36-b522-44d3-96dc-3981a41ef566 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:12.972450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953dc093-0b16-4e59-b1e1-5c4b64f4b78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:12.973413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953dc093-0b16-4e59-b1e1-5c4b64f4b78b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:12.973842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953dc093-0b16-4e59-b1e1-5c4b64f4b78b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:12.974261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953dc093-0b16-4e59-b1e1-5c4b64f4b78b None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:12.975181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953dc093-0b16-4e59-b1e1-5c4b64f4b78b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:12.975627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953dc093-0b16-4e59-b1e1-5c4b64f4b78b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:12.975751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 485/945] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:54:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:12.977411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953dc093-0b16-4e59-b1e1-5c4b64f4b78b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:13.000043 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 461/946] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 22 16:54:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:13.005354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb225e36-b522-44d3-96dc-3981a41ef566 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:13.078351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:13.128888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb225e36-b522-44d3-96dc-3981a41ef566 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:13.130395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953dc093-0b16-4e59-b1e1-5c4b64f4b78b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:13.171451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ced92-7422-41dc-a7d3-5ccf93779123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:13.172094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ced92-7422-41dc-a7d3-5ccf93779123 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:13.172373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ced92-7422-41dc-a7d3-5ccf93779123 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:13.172658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ced92-7422-41dc-a7d3-5ccf93779123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:13.187215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ced92-7422-41dc-a7d3-5ccf93779123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:13.189976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ced92-7422-41dc-a7d3-5ccf93779123 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-310ced92-7422-41dc-a7d3-5ccf93779123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:13.207488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:13.208506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:13.209641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:13.209641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:13.209641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:13.209641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 486/947] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:13.215932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ced92-7422-41dc-a7d3-5ccf93779123 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:13.256879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953dc093-0b16-4e59-b1e1-5c4b64f4b78b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:13.280278 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:13.301406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68adbd7-6cab-411a-925d-2465210a631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:13.302045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68adbd7-6cab-411a-925d-2465210a631a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:13.302322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68adbd7-6cab-411a-925d-2465210a631a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:13.302592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68adbd7-6cab-411a-925d-2465210a631a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:13.305151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68adbd7-6cab-411a-925d-2465210a631a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:13.305151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68adbd7-6cab-411a-925d-2465210a631a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f68adbd7-6cab-411a-925d-2465210a631a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:13.316578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:13.317746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:13.318646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 462/948] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 22 16:54:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:13.323710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68adbd7-6cab-411a-925d-2465210a631a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:13.342183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ced92-7422-41dc-a7d3-5ccf93779123 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:13.354772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:13.381227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185987d-4953-43c5-9966-6cb4898a0327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:13.381978 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185987d-4953-43c5-9966-6cb4898a0327 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:13.382373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185987d-4953-43c5-9966-6cb4898a0327 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:13.382736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185987d-4953-43c5-9966-6cb4898a0327 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:13.383509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185987d-4953-43c5-9966-6cb4898a0327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:13.383906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185987d-4953-43c5-9966-6cb4898a0327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:13.385896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185987d-4953-43c5-9966-6cb4898a0327 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:13.407060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 487/949] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:13.412662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185987d-4953-43c5-9966-6cb4898a0327 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:13.457859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68adbd7-6cab-411a-925d-2465210a631a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:13.512094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e58fd-5d5a-428c-98e7-9d396440c535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:13.512876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e58fd-5d5a-428c-98e7-9d396440c535 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:13.513328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e58fd-5d5a-428c-98e7-9d396440c535 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:13.513675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e58fd-5d5a-428c-98e7-9d396440c535 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:13.514639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e58fd-5d5a-428c-98e7-9d396440c535 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:13.515039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e58fd-5d5a-428c-98e7-9d396440c535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:13.516918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e58fd-5d5a-428c-98e7-9d396440c535 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:13.527049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 463/950] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 22 16:54:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:13.529761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e58fd-5d5a-428c-98e7-9d396440c535 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:13.539822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185987d-4953-43c5-9966-6cb4898a0327 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:13.579213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e38988-c3a9-418d-9c15-a64b2e1f7fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:13.579585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38988-c3a9-418d-9c15-a64b2e1f7fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:13.579871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38988-c3a9-418d-9c15-a64b2e1f7fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:13.580150 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38988-c3a9-418d-9c15-a64b2e1f7fb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:13.590003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e38988-c3a9-418d-9c15-a64b2e1f7fb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:13.592319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e38988-c3a9-418d-9c15-a64b2e1f7fb5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:13.596393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-f9e38988-c3a9-418d-9c15-a64b2e1f7fb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:54:13.609652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:13.610610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:13.611736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:54:13.611736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:54:13.611736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:54:13.611736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:54:13.611736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:54:13.611736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:54:13.611736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:13.611736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:13.615253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 488/951] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:13.626788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e38988-c3a9-418d-9c15-a64b2e1f7fb5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:13.659321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:13.662016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e58fd-5d5a-428c-98e7-9d396440c535 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:13.703254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc223b8-738a-4e35-9424-52b8f0609c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:13.703878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc223b8-738a-4e35-9424-52b8f0609c57 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:13.704160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc223b8-738a-4e35-9424-52b8f0609c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:13.704431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc223b8-738a-4e35-9424-52b8f0609c57 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:13.705073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc223b8-738a-4e35-9424-52b8f0609c57 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:13.705436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc223b8-738a-4e35-9424-52b8f0609c57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:13.706872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc223b8-738a-4e35-9424-52b8f0609c57 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:13.713135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 464/952] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 22 16:54:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:13.719926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc223b8-738a-4e35-9424-52b8f0609c57 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:13.737721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e38988-c3a9-418d-9c15-a64b2e1f7fb5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:13.784611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791d820-ea66-48e3-8cab-29b94a06980b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:13.785289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791d820-ea66-48e3-8cab-29b94a06980b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:13.785700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791d820-ea66-48e3-8cab-29b94a06980b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:13.786104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791d820-ea66-48e3-8cab-29b94a06980b None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:13.786930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791d820-ea66-48e3-8cab-29b94a06980b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:13.787390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791d820-ea66-48e3-8cab-29b94a06980b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:13.789082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791d820-ea66-48e3-8cab-29b94a06980b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:13.808640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 489/953] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:54:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:13.809814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791d820-ea66-48e3-8cab-29b94a06980b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:13.855948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc223b8-738a-4e35-9424-52b8f0609c57 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:13.868054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:13.902427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6a760-e00e-438a-a900-ebc097b8f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:13.903125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6a760-e00e-438a-a900-ebc097b8f0c0 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:13.903410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6a760-e00e-438a-a900-ebc097b8f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:13.903686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6a760-e00e-438a-a900-ebc097b8f0c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:13.904661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6a760-e00e-438a-a900-ebc097b8f0c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:13.905946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6a760-e00e-438a-a900-ebc097b8f0c0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:13.914848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:13.928986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791d820-ea66-48e3-8cab-29b94a06980b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:13.939110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 465/954] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 16:54:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:13.985335 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2fa86-6b20-4bd2-a55f-ec2b761adbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:13.986478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2fa86-6b20-4bd2-a55f-ec2b761adbf9 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:13.986986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2fa86-6b20-4bd2-a55f-ec2b761adbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:13.987558 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2fa86-6b20-4bd2-a55f-ec2b761adbf9 None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/abbbe2b3f23b4c1caf8e20825cc2cfee/roles/34a4f36a154c4c03942a6a78e370fb70` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:13.990385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2fa86-6b20-4bd2-a55f-ec2b761adbf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=abbbe2b3f23b4c1caf8e20825cc2cfee, role_id=34a4f36a154c4c03942a6a78e370fb70)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:13.992270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2fa86-6b20-4bd2-a55f-ec2b761adbf9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:14.010195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 490/955] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:54:13 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/abbbe2b3f23b4c1caf8e20825cc2cfee/roles/34a4f36a154c4c03942a6a78e370fb70 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:54:14.540341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001fc4-78da-49c6-ad9b-b2004065b843 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:14.541451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001fc4-78da-49c6-ad9b-b2004065b843 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:14.541681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001fc4-78da-49c6-ad9b-b2004065b843 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:14.543313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 466/956] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:14.549143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0453ad-4cde-4ce1-b657-2b012e320988 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:14.549428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0453ad-4cde-4ce1-b657-2b012e320988 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:14.549700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0453ad-4cde-4ce1-b657-2b012e320988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:14.568543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8d0453ad-4cde-4ce1-b657-2b012e320988 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:14.570101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8d0453ad-4cde-4ce1-b657-2b012e320988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:14.643708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 491/957] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:14.920264 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b001fc4-78da-49c6-ad9b-b2004065b843 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:15.043103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b001fc4-78da-49c6-ad9b-b2004065b843 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:15.087262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62538c5-f45b-49a2-986d-6db639327eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:15.088420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62538c5-f45b-49a2-986d-6db639327eee None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:15.088782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62538c5-f45b-49a2-986d-6db639327eee None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:15.089059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62538c5-f45b-49a2-986d-6db639327eee None admin] PATH_INFO: `/v3/services/compute` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:15.090075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62538c5-f45b-49a2-986d-6db639327eee None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:15.091855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62538c5-f45b-49a2-986d-6db639327eee None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-a62538c5-f45b-49a2-986d-6db639327eee None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:15.101977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:15.103009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 16:54:15.103939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:15.104680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 467/958] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:15.110322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0453ad-4cde-4ce1-b657-2b012e320988 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:15.196061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:15.220765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0453ad-4cde-4ce1-b657-2b012e320988 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:15.261832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c61f70-dd00-40bb-847e-513da3762488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:15.262447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61f70-dd00-40bb-847e-513da3762488 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:15.262725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61f70-dd00-40bb-847e-513da3762488 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:15.263017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61f70-dd00-40bb-847e-513da3762488 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:15.263644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c61f70-dd00-40bb-847e-513da3762488 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:15.263940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c61f70-dd00-40bb-847e-513da3762488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:15.265298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c61f70-dd00-40bb-847e-513da3762488 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:15.270762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 492/959] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:54:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:15.276777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62538c5-f45b-49a2-986d-6db639327eee None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:15.309759 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:15.394768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62538c5-f45b-49a2-986d-6db639327eee None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:15.433623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a590d8-a628-4189-8985-77edbfff9b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:15.434573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a590d8-a628-4189-8985-77edbfff9b29 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:15.435105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a590d8-a628-4189-8985-77edbfff9b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:15.435571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a590d8-a628-4189-8985-77edbfff9b29 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:15.436401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a590d8-a628-4189-8985-77edbfff9b29 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:15.436893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a590d8-a628-4189-8985-77edbfff9b29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:15.438449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a590d8-a628-4189-8985-77edbfff9b29 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:15.443961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 468/960] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:54:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:15.467020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1d14a-6bc5-44bd-8016-bd261eff089f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:15.467330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1d14a-6bc5-44bd-8016-bd261eff089f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:15.467490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1d14a-6bc5-44bd-8016-bd261eff089f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:15.469171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 493/961] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:15.475162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb2d6-b57d-4d72-a6a0-339e69fe6703 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:15.475809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb2d6-b57d-4d72-a6a0-339e69fe6703 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:15.476290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb2d6-b57d-4d72-a6a0-339e69fe6703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:15.496531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-af5bb2d6-b57d-4d72-a6a0-339e69fe6703 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:15.498169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-af5bb2d6-b57d-4d72-a6a0-339e69fe6703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:15.576636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 469/962] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:15.874468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1d14a-6bc5-44bd-8016-bd261eff089f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:15.999513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1d14a-6bc5-44bd-8016-bd261eff089f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:16.053570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f3170-633f-4208-8ccf-4967f1240e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:16.054526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f3170-633f-4208-8ccf-4967f1240e24 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:16.055013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f3170-633f-4208-8ccf-4967f1240e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:16.055508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f3170-633f-4208-8ccf-4967f1240e24 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:16.056241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f3170-633f-4208-8ccf-4967f1240e24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:16.058354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f3170-633f-4208-8ccf-4967f1240e24 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:16.078677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.sql.core [None req-3a4f3170-633f-4208-8ccf-4967f1240e24 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 16:54:16.078677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 16:54:16.078677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [parameters: {'user_id': '7faf7dfb26bf491abd7f341b295cee15', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 16:54:16.078677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81561) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-3a4f3170-633f-4208-8ccf-4967f1240e24 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application conn.query(q) Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result.read() Sep 22 16:54:16.090233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 16:54:16.091332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application next(self.gen) Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application yield resource Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application next(self.gen) Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise self.value Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application session.commit() Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 16:54:16.092305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.session.flush() Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application compat.raise_( Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 16:54:16.093274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 16:54:16.094177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application conn.query(q) Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result.read() Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 16:54:16.096028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [parameters: {'user_id': '7faf7dfb26bf491abd7f341b295cee15', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:16.097601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 22 16:54:16.098589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 22 16:54:16.099732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 22 16:54:16.099732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 22 16:54:16.099732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 16:54:16.099732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 16:54:16.099732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 16:54:16.099732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:16.099732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 494/963] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 22 16:54:15 2022] POST /identity/v3/users => generated 160 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:16.101312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bb2d6-b57d-4d72-a6a0-339e69fe6703 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:16.135343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:16.213280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bb2d6-b57d-4d72-a6a0-339e69fe6703 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:16.250906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e8d58-1cc2-4d21-9198-83d372f5f0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:16.251949 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e8d58-1cc2-4d21-9198-83d372f5f0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:16.252430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e8d58-1cc2-4d21-9198-83d372f5f0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:16.252902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e8d58-1cc2-4d21-9198-83d372f5f0a9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:16.261029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e8d58-1cc2-4d21-9198-83d372f5f0a9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:16.262854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e8d58-1cc2-4d21-9198-83d372f5f0a9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-cb1e8d58-1cc2-4d21-9198-83d372f5f0a9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:54:16.276246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:16.277240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:16.278181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:16.278181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:16.278181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:54:16.278181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:54:16.278181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:54:16.278181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:54:16.278181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:54:16.278181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:54:16.278181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 16:54:16.278181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:16.280967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 470/964] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:54:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:16.287553 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f3170-633f-4208-8ccf-4967f1240e24 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:16.292639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:16.417907 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f3170-633f-4208-8ccf-4967f1240e24 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:16.430258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4482-d5eb-4307-b454-b3fa4b452429 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:16.430567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4482-d5eb-4307-b454-b3fa4b452429 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:16.430991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4482-d5eb-4307-b454-b3fa4b452429 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:16.433178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 471/965] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:16.440002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30dd58-d129-4e37-81b3-fcd5c0a55e64 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:16.440281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30dd58-d129-4e37-81b3-fcd5c0a55e64 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:16.440556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30dd58-d129-4e37-81b3-fcd5c0a55e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:16.456544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcaefe-3355-4602-bc69-17265e270600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:16.457294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcaefe-3355-4602-bc69-17265e270600 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:16.457674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcaefe-3355-4602-bc69-17265e270600 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:16.458067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcaefe-3355-4602-bc69-17265e270600 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:16.458983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcaefe-3355-4602-bc69-17265e270600 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:16.459349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2a30dd58-d129-4e37-81b3-fcd5c0a55e64 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:16.459464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcaefe-3355-4602-bc69-17265e270600 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:16.460540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2a30dd58-d129-4e37-81b3-fcd5c0a55e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:16.461275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcaefe-3355-4602-bc69-17265e270600 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:16.502310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 495/966] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 22 16:54:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:16.504263 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc8ad-9462-491d-823a-19fdcc9e1da0 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:16.504545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc8ad-9462-491d-823a-19fdcc9e1da0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:16.504814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc8ad-9462-491d-823a-19fdcc9e1da0 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:16.511160 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 496/967] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:16.514471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef030b6-4c9b-49db-b20c-5dbbedde7f90 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:16.514750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef030b6-4c9b-49db-b20c-5dbbedde7f90 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:16.515075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef030b6-4c9b-49db-b20c-5dbbedde7f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:16.536206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3ef030b6-4c9b-49db-b20c-5dbbedde7f90 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:16.537904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3ef030b6-4c9b-49db-b20c-5dbbedde7f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:16.540241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 472/968] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:16.609270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:16.628307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 497/969] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:16.835010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30dd58-d129-4e37-81b3-fcd5c0a55e64 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:16.852758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:16.903204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef030b6-4c9b-49db-b20c-5dbbedde7f90 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:16.956912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30dd58-d129-4e37-81b3-fcd5c0a55e64 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:16.994496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d17a8-945c-47e9-99da-db0e2c5f8848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:16.995269 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d17a8-945c-47e9-99da-db0e2c5f8848 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:16.995541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d17a8-945c-47e9-99da-db0e2c5f8848 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:16.995804 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d17a8-945c-47e9-99da-db0e2c5f8848 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.006127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d17a8-945c-47e9-99da-db0e2c5f8848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-911d17a8-945c-47e9-99da-db0e2c5f8848 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:54:17.011963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:17.013483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:17.014385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 473/970] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 22 16:54:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:17.019315 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d17a8-945c-47e9-99da-db0e2c5f8848 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:17.024390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef030b6-4c9b-49db-b20c-5dbbedde7f90 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:17.070178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744a3ee-9c00-4d38-b1a2-fd5943ffeecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:17.070821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744a3ee-9c00-4d38-b1a2-fd5943ffeecf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.071129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744a3ee-9c00-4d38-b1a2-fd5943ffeecf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.071403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744a3ee-9c00-4d38-b1a2-fd5943ffeecf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.072564 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744a3ee-9c00-4d38-b1a2-fd5943ffeecf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:17.073915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744a3ee-9c00-4d38-b1a2-fd5943ffeecf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a744a3ee-9c00-4d38-b1a2-fd5943ffeecf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:17.083167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:17.084264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:17.085128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:17.091122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 498/971] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 22 16:54:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:17.095537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744a3ee-9c00-4d38-b1a2-fd5943ffeecf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:17.114360 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:17.145263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d17a8-945c-47e9-99da-db0e2c5f8848 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:17.157500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:17.181135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e755e-909c-4936-acd8-3aacf53a3d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:17.181769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e755e-909c-4936-acd8-3aacf53a3d5f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.182467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e755e-909c-4936-acd8-3aacf53a3d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.182805 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e755e-909c-4936-acd8-3aacf53a3d5f None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.183502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e755e-909c-4936-acd8-3aacf53a3d5f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:17.183804 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e755e-909c-4936-acd8-3aacf53a3d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:17.185879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e755e-909c-4936-acd8-3aacf53a3d5f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:17.203762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 474/972] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 22 16:54:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:17.205913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e755e-909c-4936-acd8-3aacf53a3d5f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:17.211448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744a3ee-9c00-4d38-b1a2-fd5943ffeecf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:17.251532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09327d09-4be1-4dbc-85ca-cee2d2445515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:17.252302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09327d09-4be1-4dbc-85ca-cee2d2445515 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.252572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09327d09-4be1-4dbc-85ca-cee2d2445515 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.252837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09327d09-4be1-4dbc-85ca-cee2d2445515 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.253475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09327d09-4be1-4dbc-85ca-cee2d2445515 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:17.253760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09327d09-4be1-4dbc-85ca-cee2d2445515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:17.255164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09327d09-4be1-4dbc-85ca-cee2d2445515 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:17.261190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 499/973] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 22 16:54:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:17.267157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09327d09-4be1-4dbc-85ca-cee2d2445515 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:17.332905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e755e-909c-4936-acd8-3aacf53a3d5f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:17.371386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01f0fb-1934-4d2e-8ade-c5e76e63ed6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:17.372024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01f0fb-1934-4d2e-8ade-c5e76e63ed6d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.372306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01f0fb-1934-4d2e-8ade-c5e76e63ed6d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.372578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01f0fb-1934-4d2e-8ade-c5e76e63ed6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.376450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:17.386578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09327d09-4be1-4dbc-85ca-cee2d2445515 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:17.388433 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01f0fb-1934-4d2e-8ade-c5e76e63ed6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:17.390038 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01f0fb-1934-4d2e-8ade-c5e76e63ed6d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:17.405725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-ab01f0fb-1934-4d2e-8ade-c5e76e63ed6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:17.406906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:17.407930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:17.414056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:17.414056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:17.414056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:17.414056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 475/974] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:17.414056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01f0fb-1934-4d2e-8ade-c5e76e63ed6d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:17.431927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ba03c-45a4-4298-b570-9e15fdd414a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:17.432537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ba03c-45a4-4298-b570-9e15fdd414a0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.432816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ba03c-45a4-4298-b570-9e15fdd414a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.433084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ba03c-45a4-4298-b570-9e15fdd414a0 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.433704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ba03c-45a4-4298-b570-9e15fdd414a0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:17.434004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ba03c-45a4-4298-b570-9e15fdd414a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:17.435384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ba03c-45a4-4298-b570-9e15fdd414a0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:17.440155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 500/975] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 22 16:54:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:17.446602 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ba03c-45a4-4298-b570-9e15fdd414a0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:17.544397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01f0fb-1934-4d2e-8ade-c5e76e63ed6d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:17.563686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ba03c-45a4-4298-b570-9e15fdd414a0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:17.581111 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771650c8-c536-4251-89a9-fa3f4054f47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:17.581706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771650c8-c536-4251-89a9-fa3f4054f47b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.581983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771650c8-c536-4251-89a9-fa3f4054f47b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.582247 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771650c8-c536-4251-89a9-fa3f4054f47b None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.583042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771650c8-c536-4251-89a9-fa3f4054f47b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:17.583343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771650c8-c536-4251-89a9-fa3f4054f47b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:17.584697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771650c8-c536-4251-89a9-fa3f4054f47b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:17.597522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 476/976] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:17.602971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771650c8-c536-4251-89a9-fa3f4054f47b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:17.603825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbca82-ddfc-4fc8-8781-ee1d88e18f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:17.604796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbca82-ddfc-4fc8-8781-ee1d88e18f4f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.605297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbca82-ddfc-4fc8-8781-ee1d88e18f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.605770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbca82-ddfc-4fc8-8781-ee1d88e18f4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.606501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbca82-ddfc-4fc8-8781-ee1d88e18f4f None admin] RBAC: Adding query filter params (interface=admin, service_id=8f7eb0cc6fd24a668b485d996afe8f7b, region_id=RegionOne) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:17.606938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbca82-ddfc-4fc8-8781-ee1d88e18f4f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:17.610471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbca82-ddfc-4fc8-8781-ee1d88e18f4f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:17.618064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 501/977] 10.222.0.16 () {56 vars in 1249 bytes} [Thu Sep 22 16:54:17 2022] GET /identity/v3/endpoints?service_id=8f7eb0cc6fd24a668b485d996afe8f7b&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:17.624015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbca82-ddfc-4fc8-8781-ee1d88e18f4f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:17.667848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:17.681388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:17.729663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771650c8-c536-4251-89a9-fa3f4054f47b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:17.743374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbca82-ddfc-4fc8-8781-ee1d88e18f4f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:17.773682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862f2c9-bcba-4887-94b6-88ab0b671dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:17.774321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862f2c9-bcba-4887-94b6-88ab0b671dbb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.774600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862f2c9-bcba-4887-94b6-88ab0b671dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.774871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862f2c9-bcba-4887-94b6-88ab0b671dbb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.782437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00806893-08f1-42d9-b8b6-f84b2a7161e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:17.782761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862f2c9-bcba-4887-94b6-88ab0b671dbb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:17.783057 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00806893-08f1-42d9-b8b6-f84b2a7161e2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.783337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00806893-08f1-42d9-b8b6-f84b2a7161e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.783603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00806893-08f1-42d9-b8b6-f84b2a7161e2 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.784189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00806893-08f1-42d9-b8b6-f84b2a7161e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:17.784291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862f2c9-bcba-4887-94b6-88ab0b671dbb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:17.785647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00806893-08f1-42d9-b8b6-f84b2a7161e2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:17.791539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 502/978] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 22 16:54:17 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 16:54:17.794492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738a542-ddcd-4858-814a-835c7d43aba5 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.794711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738a542-ddcd-4858-814a-835c7d43aba5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.794994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738a542-ddcd-4858-814a-835c7d43aba5 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-e862f2c9-bcba-4887-94b6-88ab0b671dbb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:54:17.796126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:17.799877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:17.800847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:17.800847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:17.800847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:54:17.800847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:54:17.800847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:54:17.800847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:54:17.800847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:54:17.800847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:54:17.800847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 16:54:17.800847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:17.801400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 503/979] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:17.801477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 477/980] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 22 16:54:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:17.803479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3f4b-3441-48de-b340-1aad0c2adcbb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.803840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3f4b-3441-48de-b340-1aad0c2adcbb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.804124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3f4b-3441-48de-b340-1aad0c2adcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.805408 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862f2c9-bcba-4887-94b6-88ab0b671dbb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:17.828313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-4cfb3f4b-3441-48de-b340-1aad0c2adcbb None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:17.829568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4cfb3f4b-3441-48de-b340-1aad0c2adcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:17.918292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 504/981] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:17.925597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862f2c9-bcba-4887-94b6-88ab0b671dbb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:17.937239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:17.969214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5d7fa-d00f-42a6-b4c8-574d2faf3656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:17.969836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5d7fa-d00f-42a6-b4c8-574d2faf3656 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:17.970117 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5d7fa-d00f-42a6-b4c8-574d2faf3656 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:17.970395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5d7fa-d00f-42a6-b4c8-574d2faf3656 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:17.971255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5d7fa-d00f-42a6-b4c8-574d2faf3656 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:17.971566 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5d7fa-d00f-42a6-b4c8-574d2faf3656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:17.973801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5d7fa-d00f-42a6-b4c8-574d2faf3656 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:17.993447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 478/982] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 22 16:54:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:18.000122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb3f4b-3441-48de-b340-1aad0c2adcbb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:18.045521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:18.108697 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb3f4b-3441-48de-b340-1aad0c2adcbb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:18.144937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85955d40-f559-433c-9672-3a2af68b9b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:18.145547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85955d40-f559-433c-9672-3a2af68b9b38 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:18.145827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85955d40-f559-433c-9672-3a2af68b9b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:18.146099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85955d40-f559-433c-9672-3a2af68b9b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:18.192981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5d7fa-d00f-42a6-b4c8-574d2faf3656 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:18.218701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85955d40-f559-433c-9672-3a2af68b9b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:18.220180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85955d40-f559-433c-9672-3a2af68b9b38 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:18.266718 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:18.312344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5d7fa-d00f-42a6-b4c8-574d2faf3656 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:18.333252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 505/983] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:54:17 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:54:18.339441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85955d40-f559-433c-9672-3a2af68b9b38 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:18.351555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53d90a-38bb-4a40-9f76-49e5fd65078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:18.352158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53d90a-38bb-4a40-9f76-49e5fd65078e None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:18.352434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53d90a-38bb-4a40-9f76-49e5fd65078e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:18.352703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53d90a-38bb-4a40-9f76-49e5fd65078e None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:18.353933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe53d90a-38bb-4a40-9f76-49e5fd65078e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:18.355349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe53d90a-38bb-4a40-9f76-49e5fd65078e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:18.370882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 479/984] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:54:18 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 16:54:18.455474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85955d40-f559-433c-9672-3a2af68b9b38 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:18.500811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0afdfa-7ef6-4c14-a25e-b209c4a0c744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:18.501417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0afdfa-7ef6-4c14-a25e-b209c4a0c744 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:18.501693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0afdfa-7ef6-4c14-a25e-b209c4a0c744 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:18.501961 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0afdfa-7ef6-4c14-a25e-b209c4a0c744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:18.505116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:18.515399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0afdfa-7ef6-4c14-a25e-b209c4a0c744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:18.516863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0afdfa-7ef6-4c14-a25e-b209c4a0c744 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-ec0afdfa-7ef6-4c14-a25e-b209c4a0c744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:18.532809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:18.533867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:18.539720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:18.539720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:18.539720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:18.539720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 506/985] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:18.541216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53d90a-38bb-4a40-9f76-49e5fd65078e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:18.571899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:18.647878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53d90a-38bb-4a40-9f76-49e5fd65078e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:18.683680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930a73c-15bd-42e3-92f4-90dfb9e08686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:18.684594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930a73c-15bd-42e3-92f4-90dfb9e08686 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:18.685069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930a73c-15bd-42e3-92f4-90dfb9e08686 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:18.685519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930a73c-15bd-42e3-92f4-90dfb9e08686 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:18.686259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930a73c-15bd-42e3-92f4-90dfb9e08686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:18.686567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930a73c-15bd-42e3-92f4-90dfb9e08686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:18.687978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930a73c-15bd-42e3-92f4-90dfb9e08686 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:18.700729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 480/986] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:18.706772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0afdfa-7ef6-4c14-a25e-b209c4a0c744 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:18.822784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0afdfa-7ef6-4c14-a25e-b209c4a0c744 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:18.858051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8e7cb-dae4-4639-8047-168a298922ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:18.858989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8e7cb-dae4-4639-8047-168a298922ee None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:18.859454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8e7cb-dae4-4639-8047-168a298922ee None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:18.859940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8e7cb-dae4-4639-8047-168a298922ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:18.866892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8e7cb-dae4-4639-8047-168a298922ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:18.868565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8e7cb-dae4-4639-8047-168a298922ee None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-0da8e7cb-dae4-4639-8047-168a298922ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:54:18.880258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:18.881344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:18.882462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:54:18.882462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:54:18.882462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:54:18.882462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:54:18.882462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:54:18.882462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:54:18.882462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:18.882462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:18.884565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 507/987] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:18.890149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930a73c-15bd-42e3-92f4-90dfb9e08686 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:18.978729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:18.995333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930a73c-15bd-42e3-92f4-90dfb9e08686 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:19.030546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70be4df-b716-41b8-b1ea-a287cc157c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:19.031171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70be4df-b716-41b8-b1ea-a287cc157c6b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:19.031446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70be4df-b716-41b8-b1ea-a287cc157c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:19.031712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70be4df-b716-41b8-b1ea-a287cc157c6b None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:19.032391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70be4df-b716-41b8-b1ea-a287cc157c6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:19.032690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70be4df-b716-41b8-b1ea-a287cc157c6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:19.034095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70be4df-b716-41b8-b1ea-a287cc157c6b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:19.051412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 481/988] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:54:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:19.052588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8e7cb-dae4-4639-8047-168a298922ee None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:19.103176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:19.157831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8e7cb-dae4-4639-8047-168a298922ee None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:19.192667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede21db-79e0-4f74-926b-e4b235959e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:19.193333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede21db-79e0-4f74-926b-e4b235959e21 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:19.193619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede21db-79e0-4f74-926b-e4b235959e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:19.193890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede21db-79e0-4f74-926b-e4b235959e21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:19.194610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede21db-79e0-4f74-926b-e4b235959e21 None admin] RBAC: Adding query filter params (role.id=34a4f36a154c4c03942a6a78e370fb70, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=abbbe2b3f23b4c1caf8e20825cc2cfee) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:19.194918 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede21db-79e0-4f74-926b-e4b235959e21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:19.196491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede21db-79e0-4f74-926b-e4b235959e21 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:19.211275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 508/989] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:54:19 2022] GET /identity/v3/role_assignments?user.id=abbbe2b3f23b4c1caf8e20825cc2cfee&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=34a4f36a154c4c03942a6a78e370fb70 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:20.012807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a780023-cb51-4442-8f98-8fae3362fe64 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:20.013440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a780023-cb51-4442-8f98-8fae3362fe64 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:20.013440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a780023-cb51-4442-8f98-8fae3362fe64 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:20.014823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 482/990] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:54:20.020092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e307a07-7708-4211-bbc8-a74ec45c58d4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:20.020532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e307a07-7708-4211-bbc8-a74ec45c58d4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:20.020629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e307a07-7708-4211-bbc8-a74ec45c58d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:20.039672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7e307a07-7708-4211-bbc8-a74ec45c58d4 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:20.041384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7e307a07-7708-4211-bbc8-a74ec45c58d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:20.110655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 509/991] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:20.378412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a780023-cb51-4442-8f98-8fae3362fe64 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:20.486796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a780023-cb51-4442-8f98-8fae3362fe64 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:20.522393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187edddf-ad85-4a66-a988-097aa7be3812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:20.523046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187edddf-ad85-4a66-a988-097aa7be3812 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:20.523327 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187edddf-ad85-4a66-a988-097aa7be3812 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:20.523597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187edddf-ad85-4a66-a988-097aa7be3812 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:20.524179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187edddf-ad85-4a66-a988-097aa7be3812 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:20.525659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187edddf-ad85-4a66-a988-097aa7be3812 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-187edddf-ad85-4a66-a988-097aa7be3812 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:20.534604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:20.535685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 16:54:20.536577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:20.539062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 483/992] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 22 16:54:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:20.542333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e307a07-7708-4211-bbc8-a74ec45c58d4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:20.581639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de557f-0fbb-4060-b115-2628f7f96f48 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:20.581949 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de557f-0fbb-4060-b115-2628f7f96f48 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:20.582172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de557f-0fbb-4060-b115-2628f7f96f48 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:20.583722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 484/993] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:54:20.587965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:20.588531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0777414-ed99-4727-839c-6af2f1bdb34d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:20.588814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0777414-ed99-4727-839c-6af2f1bdb34d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:20.589088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0777414-ed99-4727-839c-6af2f1bdb34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:20.606561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d0777414-ed99-4727-839c-6af2f1bdb34d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:20.607756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d0777414-ed99-4727-839c-6af2f1bdb34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:20.610575 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:20.649901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e307a07-7708-4211-bbc8-a74ec45c58d4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:20.677871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 485/994] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:20.686163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a24e21-efc8-4f7c-b299-64a2594b5797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:20.686764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a24e21-efc8-4f7c-b299-64a2594b5797 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:20.687078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a24e21-efc8-4f7c-b299-64a2594b5797 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:20.687346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a24e21-efc8-4f7c-b299-64a2594b5797 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:20.687964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a24e21-efc8-4f7c-b299-64a2594b5797 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:20.688258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a24e21-efc8-4f7c-b299-64a2594b5797 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:20.689593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a24e21-efc8-4f7c-b299-64a2594b5797 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:20.698840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 510/995] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 22 16:54:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:20.700386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0777414-ed99-4727-839c-6af2f1bdb34d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:20.810620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0777414-ed99-4727-839c-6af2f1bdb34d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:20.848861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02369a-d4cb-43f6-ba14-912739371657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:20.849471 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02369a-d4cb-43f6-ba14-912739371657 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:20.849752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02369a-d4cb-43f6-ba14-912739371657 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:20.850022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02369a-d4cb-43f6-ba14-912739371657 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:20.850650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02369a-d4cb-43f6-ba14-912739371657 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:20.850980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02369a-d4cb-43f6-ba14-912739371657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:20.852335 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02369a-d4cb-43f6-ba14-912739371657 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:20.857636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 486/996] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 22 16:54:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:20.863191 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a24e21-efc8-4f7c-b299-64a2594b5797 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:20.952367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02369a-d4cb-43f6-ba14-912739371657 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:20.956148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:20.966782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a24e21-efc8-4f7c-b299-64a2594b5797 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:20.974719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:21.005804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708aed7-a4d2-4aa2-8be7-ac8f968d615b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:21.006417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708aed7-a4d2-4aa2-8be7-ac8f968d615b None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:21.006696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708aed7-a4d2-4aa2-8be7-ac8f968d615b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:21.006987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708aed7-a4d2-4aa2-8be7-ac8f968d615b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:21.007664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708aed7-a4d2-4aa2-8be7-ac8f968d615b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:21.009025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708aed7-a4d2-4aa2-8be7-ac8f968d615b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:21.027154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 511/997] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 16:54:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:21.071444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02369a-d4cb-43f6-ba14-912739371657 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:21.117982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c21d1-2bd1-42ea-b537-609fc20845e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:21.118714 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c21d1-2bd1-42ea-b537-609fc20845e8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:21.119002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c21d1-2bd1-42ea-b537-609fc20845e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:21.119277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c21d1-2bd1-42ea-b537-609fc20845e8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:21.120013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c21d1-2bd1-42ea-b537-609fc20845e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:21.121272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c21d1-2bd1-42ea-b537-609fc20845e8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c53c21d1-2bd1-42ea-b537-609fc20845e8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:21.131178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:21.132429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:21.138338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 487/998] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:21.139231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708aed7-a4d2-4aa2-8be7-ac8f968d615b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:21.255086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708aed7-a4d2-4aa2-8be7-ac8f968d615b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:21.289491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1db9f-7463-47c0-95bf-1bb4fe6496c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:21.290099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1db9f-7463-47c0-95bf-1bb4fe6496c0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:21.290380 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1db9f-7463-47c0-95bf-1bb4fe6496c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:21.291242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1db9f-7463-47c0-95bf-1bb4fe6496c0 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:21.291867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1db9f-7463-47c0-95bf-1bb4fe6496c0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:21.292169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1db9f-7463-47c0-95bf-1bb4fe6496c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:21.293526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1db9f-7463-47c0-95bf-1bb4fe6496c0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:21.302991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 512/999] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:54:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:21.304195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c21d1-2bd1-42ea-b537-609fc20845e8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:21.374316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:21.414243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c21d1-2bd1-42ea-b537-609fc20845e8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:21.455300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cf79e-d609-4f23-b2ee-9b6b6868f6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:21.455920 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cf79e-d609-4f23-b2ee-9b6b6868f6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:21.456198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cf79e-d609-4f23-b2ee-9b6b6868f6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:21.456468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cf79e-d609-4f23-b2ee-9b6b6868f6b2 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:21.456977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df770b5-f0b7-4674-8ed9-45c7774b5e1b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:21.457089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cf79e-d609-4f23-b2ee-9b6b6868f6b2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:21.457384 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cf79e-d609-4f23-b2ee-9b6b6868f6b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:21.457727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df770b5-f0b7-4674-8ed9-45c7774b5e1b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:21.458187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df770b5-f0b7-4674-8ed9-45c7774b5e1b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:21.458746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cf79e-d609-4f23-b2ee-9b6b6868f6b2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:21.460081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 513/1000] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:54:21.465962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4999e-cba9-49ba-8a28-0f5b243416db None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:21.466240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4999e-cba9-49ba-8a28-0f5b243416db None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:21.466512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4999e-cba9-49ba-8a28-0f5b243416db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:21.466614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 488/1001] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:54:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:21.472610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cf79e-d609-4f23-b2ee-9b6b6868f6b2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:21.486907 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ebe4999e-cba9-49ba-8a28-0f5b243416db None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:21.488130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ebe4999e-cba9-49ba-8a28-0f5b243416db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:21.514279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:21.563499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 514/1002] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:21.584975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cf79e-d609-4f23-b2ee-9b6b6868f6b2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:21.612653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:21.622002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd50e92-9e31-4c9f-908d-1e0527e43a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:21.622604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd50e92-9e31-4c9f-908d-1e0527e43a83 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:21.622878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd50e92-9e31-4c9f-908d-1e0527e43a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:21.623181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd50e92-9e31-4c9f-908d-1e0527e43a83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:21.623869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd50e92-9e31-4c9f-908d-1e0527e43a83 None admin] RBAC: Adding query filter params (interface=public, service_id=2318815ebfbd437da26bd713a854cfa5, region_id=RegionOne) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:21.624171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd50e92-9e31-4c9f-908d-1e0527e43a83 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:21.625526 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd50e92-9e31-4c9f-908d-1e0527e43a83 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:21.635328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 489/1003] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 22 16:54:21 2022] GET /identity/v3/endpoints?service_id=2318815ebfbd437da26bd713a854cfa5&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:21.636837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4999e-cba9-49ba-8a28-0f5b243416db None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:21.736644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4999e-cba9-49ba-8a28-0f5b243416db None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:21.777763 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbf042-7f1c-4b53-9c94-e3ef6c8ee5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:21.778305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbf042-7f1c-4b53-9c94-e3ef6c8ee5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:21.778580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbf042-7f1c-4b53-9c94-e3ef6c8ee5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:21.778850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbf042-7f1c-4b53-9c94-e3ef6c8ee5c9 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:21.779482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bbf042-7f1c-4b53-9c94-e3ef6c8ee5c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:21.780831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bbf042-7f1c-4b53-9c94-e3ef6c8ee5c9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:21.795288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 515/1004] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 22 16:54:21 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 16:54:21.829146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd50e92-9e31-4c9f-908d-1e0527e43a83 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:21.940376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd50e92-9e31-4c9f-908d-1e0527e43a83 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:21.977408 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8a903-81a5-4750-bfcb-d0c88cb4f3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:21.978094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8a903-81a5-4750-bfcb-d0c88cb4f3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:21.978371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8a903-81a5-4750-bfcb-d0c88cb4f3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:21.978642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8a903-81a5-4750-bfcb-d0c88cb4f3b7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:21.979250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8a903-81a5-4750-bfcb-d0c88cb4f3b7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:21.980750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8a903-81a5-4750-bfcb-d0c88cb4f3b7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-2ad8a903-81a5-4750-bfcb-d0c88cb4f3b7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:21.990871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:21.991900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:21.992736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 490/1005] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Sep 22 16:54:21 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:21.998503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbf042-7f1c-4b53-9c94-e3ef6c8ee5c9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:22.027013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:22.105340 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbf042-7f1c-4b53-9c94-e3ef6c8ee5c9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:22.140157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1df1ac-548a-4403-be06-2a1923ae108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:22.140762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1df1ac-548a-4403-be06-2a1923ae108a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:22.141041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1df1ac-548a-4403-be06-2a1923ae108a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:22.141310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1df1ac-548a-4403-be06-2a1923ae108a None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:22.141939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1df1ac-548a-4403-be06-2a1923ae108a None admin] RBAC: Adding query filter params (name=image) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:22.142237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1df1ac-548a-4403-be06-2a1923ae108a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:22.143615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1df1ac-548a-4403-be06-2a1923ae108a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:22.153075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 516/1006] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:22.154609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8a903-81a5-4750-bfcb-d0c88cb4f3b7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:22.214548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:22.265153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8a903-81a5-4750-bfcb-d0c88cb4f3b7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:22.300436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e402abe-b673-4af7-a738-bd9525aba608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:22.301044 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e402abe-b673-4af7-a738-bd9525aba608 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:22.301323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e402abe-b673-4af7-a738-bd9525aba608 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:22.301596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e402abe-b673-4af7-a738-bd9525aba608 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:22.302228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e402abe-b673-4af7-a738-bd9525aba608 None admin] RBAC: Adding query filter params (type=image) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:22.302530 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e402abe-b673-4af7-a738-bd9525aba608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:22.304014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e402abe-b673-4af7-a738-bd9525aba608 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:22.314139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 491/1007] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:22 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:23.219564 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b80ed-e6d6-49d4-9dbd-19b12cdadd7d None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:23.220120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b80ed-e6d6-49d4-9dbd-19b12cdadd7d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:23.220120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b80ed-e6d6-49d4-9dbd-19b12cdadd7d None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:23.221420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 517/1008] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:23.226843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe39713-c714-4ee8-b8d0-5094fa6650a1 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:23.227274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe39713-c714-4ee8-b8d0-5094fa6650a1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:23.227443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe39713-c714-4ee8-b8d0-5094fa6650a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:23.246547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bfe39713-c714-4ee8-b8d0-5094fa6650a1 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:23.247782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bfe39713-c714-4ee8-b8d0-5094fa6650a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:23.317229 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 492/1009] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:23.592154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b80ed-e6d6-49d4-9dbd-19b12cdadd7d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:23.683097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:23.696932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b80ed-e6d6-49d4-9dbd-19b12cdadd7d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:23.730497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204fccf-c9a8-4e0e-8e06-4130ec257307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:23.731155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204fccf-c9a8-4e0e-8e06-4130ec257307 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:23.731430 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204fccf-c9a8-4e0e-8e06-4130ec257307 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:23.731690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204fccf-c9a8-4e0e-8e06-4130ec257307 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:23.732800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204fccf-c9a8-4e0e-8e06-4130ec257307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:23.734204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204fccf-c9a8-4e0e-8e06-4130ec257307 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:23.738991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 518/1010] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:23.741497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe39713-c714-4ee8-b8d0-5094fa6650a1 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:23.796698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:23.850774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe39713-c714-4ee8-b8d0-5094fa6650a1 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:23.890626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292dde8-bd60-4f6b-8c78-03f5e79cd9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:23.891302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292dde8-bd60-4f6b-8c78-03f5e79cd9eb None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:23.891588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292dde8-bd60-4f6b-8c78-03f5e79cd9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:23.891864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292dde8-bd60-4f6b-8c78-03f5e79cd9eb None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:23.892537 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292dde8-bd60-4f6b-8c78-03f5e79cd9eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:23.894058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292dde8-bd60-4f6b-8c78-03f5e79cd9eb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:23.924155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 493/1011] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:54:23 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:24.002490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e31323-8890-491a-b68d-eb1f2ca84b3b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:24.002839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e31323-8890-491a-b68d-eb1f2ca84b3b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:24.003090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e31323-8890-491a-b68d-eb1f2ca84b3b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:24.004679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 519/1012] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:24.011019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40cd65-b5f2-40cd-878f-81033108edc1 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:24.011356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40cd65-b5f2-40cd-878f-81033108edc1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:24.011582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40cd65-b5f2-40cd-878f-81033108edc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:24.031251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-fe40cd65-b5f2-40cd-878f-81033108edc1 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:24.032476 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fe40cd65-b5f2-40cd-878f-81033108edc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:24.109241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 494/1013] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:24 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:24.380797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e31323-8890-491a-b68d-eb1f2ca84b3b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:24.491145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e31323-8890-491a-b68d-eb1f2ca84b3b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:24.526067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48a31e-b2a3-4595-8aea-c29413f12bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:24.526753 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48a31e-b2a3-4595-8aea-c29413f12bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:24.526989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48a31e-b2a3-4595-8aea-c29413f12bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:24.527328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48a31e-b2a3-4595-8aea-c29413f12bc8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:24.527836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48a31e-b2a3-4595-8aea-c29413f12bc8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:24.529186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48a31e-b2a3-4595-8aea-c29413f12bc8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-4b48a31e-b2a3-4595-8aea-c29413f12bc8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:24.538011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:24.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 16:54:24.542412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:24.545787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beb645-7d21-45a4-9862-dd0da7a2ff74 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:24.545787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beb645-7d21-45a4-9862-dd0da7a2ff74 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:24.545787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beb645-7d21-45a4-9862-dd0da7a2ff74 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:24.545787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 495/1014] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:24.546778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 520/1015] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:24.546914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9beb645-7d21-45a4-9862-dd0da7a2ff74 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:24.552972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256870a1-c41d-4150-a34b-7914de812146 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:24.553256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256870a1-c41d-4150-a34b-7914de812146 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:24.553530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256870a1-c41d-4150-a34b-7914de812146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:24.574704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-256870a1-c41d-4150-a34b-7914de812146 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:24.576436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-256870a1-c41d-4150-a34b-7914de812146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:24.578871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:24.595742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:24.652988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 521/1016] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:24 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:24.659078 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9beb645-7d21-45a4-9862-dd0da7a2ff74 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:24.696856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95d0c9-35f5-442e-9427-6bba81b2ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:24.697458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95d0c9-35f5-442e-9427-6bba81b2ee92 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:24.697731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95d0c9-35f5-442e-9427-6bba81b2ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:24.697998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95d0c9-35f5-442e-9427-6bba81b2ee92 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:24.698617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95d0c9-35f5-442e-9427-6bba81b2ee92 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:24.698912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95d0c9-35f5-442e-9427-6bba81b2ee92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:24.700294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95d0c9-35f5-442e-9427-6bba81b2ee92 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:24.705321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 496/1017] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:54:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:24.710877 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256870a1-c41d-4150-a34b-7914de812146 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:24.830711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256870a1-c41d-4150-a34b-7914de812146 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:24.865348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f24e1-9189-4cb0-8e01-73933876385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:24.865949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f24e1-9189-4cb0-8e01-73933876385a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:24.866225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f24e1-9189-4cb0-8e01-73933876385a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:24.866509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f24e1-9189-4cb0-8e01-73933876385a None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:24.867149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f24e1-9189-4cb0-8e01-73933876385a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:24.867449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f24e1-9189-4cb0-8e01-73933876385a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:24.869403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f24e1-9189-4cb0-8e01-73933876385a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:24.874328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 522/1018] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 22 16:54:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:24.879919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95d0c9-35f5-442e-9427-6bba81b2ee92 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:24.932246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f24e1-9189-4cb0-8e01-73933876385a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:24.949443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:24.981288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:24.988307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95d0c9-35f5-442e-9427-6bba81b2ee92 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:25.025793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54d100-0d02-4e52-8145-ff33e6c26b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:25.026394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54d100-0d02-4e52-8145-ff33e6c26b77 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:25.026670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54d100-0d02-4e52-8145-ff33e6c26b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:25.026939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54d100-0d02-4e52-8145-ff33e6c26b77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:25.027631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54d100-0d02-4e52-8145-ff33e6c26b77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:25.028984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54d100-0d02-4e52-8145-ff33e6c26b77 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:25.041693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f24e1-9189-4cb0-8e01-73933876385a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:25.049260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 497/1019] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 16:54:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:25.091754 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17ba18-91a7-45c6-b985-83dc5e7bd34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:25.092691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17ba18-91a7-45c6-b985-83dc5e7bd34d None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:25.093190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17ba18-91a7-45c6-b985-83dc5e7bd34d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:25.093610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17ba18-91a7-45c6-b985-83dc5e7bd34d None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:25.094415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17ba18-91a7-45c6-b985-83dc5e7bd34d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:25.095989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17ba18-91a7-45c6-b985-83dc5e7bd34d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:25.115367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 523/1020] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:54:24 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 16:54:26.233558 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30d7d-372c-424f-aa3c-b49b2e612f29 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:26.234093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30d7d-372c-424f-aa3c-b49b2e612f29 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:26.234093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30d7d-372c-424f-aa3c-b49b2e612f29 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:26.235513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 498/1021] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:26.240844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504fa53-a810-42a2-80fe-f27a907510fb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:26.241279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504fa53-a810-42a2-80fe-f27a907510fb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:26.241452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504fa53-a810-42a2-80fe-f27a907510fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:26.261023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7504fa53-a810-42a2-80fe-f27a907510fb None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:26.262225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7504fa53-a810-42a2-80fe-f27a907510fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:26.324361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:26.336954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 524/1022] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:26.620908 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e30d7d-372c-424f-aa3c-b49b2e612f29 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:26.724797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e30d7d-372c-424f-aa3c-b49b2e612f29 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:26.762694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fcef2-deea-44d5-b179-eba91ba05cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:26.763686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fcef2-deea-44d5-b179-eba91ba05cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:26.764424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fcef2-deea-44d5-b179-eba91ba05cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:26.764424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fcef2-deea-44d5-b179-eba91ba05cf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:26.765554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fcef2-deea-44d5-b179-eba91ba05cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:26.766999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fcef2-deea-44d5-b179-eba91ba05cf0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:26.768881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 499/1023] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:26.774489 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504fa53-a810-42a2-80fe-f27a907510fb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:26.892242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504fa53-a810-42a2-80fe-f27a907510fb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:26.926741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e69f65-0c5e-472a-8834-d0c1ee096e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:26.927408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e69f65-0c5e-472a-8834-d0c1ee096e92 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:26.927695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e69f65-0c5e-472a-8834-d0c1ee096e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:26.927945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e69f65-0c5e-472a-8834-d0c1ee096e92 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:26.928598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e69f65-0c5e-472a-8834-d0c1ee096e92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:26.930000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e69f65-0c5e-472a-8834-d0c1ee096e92 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:26.953063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 525/1024] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:54:26 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:27.360749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947775f-fe68-47e2-904c-7a427e4213d1 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:27.361279 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947775f-fe68-47e2-904c-7a427e4213d1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:27.361279 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947775f-fe68-47e2-904c-7a427e4213d1 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:27.362819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 500/1025] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:27.369230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173dbd7-28a7-4f30-886a-957361434f24 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:27.369768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173dbd7-28a7-4f30-886a-957361434f24 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:27.369846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173dbd7-28a7-4f30-886a-957361434f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:27.389058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a173dbd7-28a7-4f30-886a-957361434f24 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:27.390267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a173dbd7-28a7-4f30-886a-957361434f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:27.397124 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc24ce-5f20-403a-9dc2-df4d595f4674 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:27.397413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc24ce-5f20-403a-9dc2-df4d595f4674 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:27.397691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc24ce-5f20-403a-9dc2-df4d595f4674 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:27.399229 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 501/1026] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:27.404337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80952252-f782-4651-b10d-6a3c3c77e080 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:27.404621 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80952252-f782-4651-b10d-6a3c3c77e080 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:27.404906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80952252-f782-4651-b10d-6a3c3c77e080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:27.423884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-80952252-f782-4651-b10d-6a3c3c77e080 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:27.425066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-80952252-f782-4651-b10d-6a3c3c77e080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:27.440507 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:27.447177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:27.468524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 526/1027] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:27.512636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 502/1028] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:27.756037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173dbd7-28a7-4f30-886a-957361434f24 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:27.785539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80952252-f782-4651-b10d-6a3c3c77e080 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:27.890326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173dbd7-28a7-4f30-886a-957361434f24 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:27.914019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80952252-f782-4651-b10d-6a3c3c77e080 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:27.932100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a094c6-713b-4ead-b5d0-574e2c291c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:27.932797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a094c6-713b-4ead-b5d0-574e2c291c6b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:27.933076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a094c6-713b-4ead-b5d0-574e2c291c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:27.933355 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a094c6-713b-4ead-b5d0-574e2c291c6b None admin] PATH_INFO: `/v3/services/image` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:27.934019 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a094c6-713b-4ead-b5d0-574e2c291c6b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:27.935421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a094c6-713b-4ead-b5d0-574e2c291c6b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e5a094c6-713b-4ead-b5d0-574e2c291c6b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:27.949387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:27.951304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:27.952903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 527/1029] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Sep 22 16:54:27 2022] GET /identity/v3/services/image => generated 86 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:27.956687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a094c6-713b-4ead-b5d0-574e2c291c6b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:27.961505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d835cec-2b6d-43bd-9a58-c342fc5a7dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:27.962025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d835cec-2b6d-43bd-9a58-c342fc5a7dab None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:27.962306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d835cec-2b6d-43bd-9a58-c342fc5a7dab None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:27.962560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d835cec-2b6d-43bd-9a58-c342fc5a7dab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:27.975016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d835cec-2b6d-43bd-9a58-c342fc5a7dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3d835cec-2b6d-43bd-9a58-c342fc5a7dab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:54:27.979919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:27.981641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:27.983270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 503/1030] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 22 16:54:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:27.987372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d835cec-2b6d-43bd-9a58-c342fc5a7dab None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:28.011030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:28.013137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:28.073606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a094c6-713b-4ead-b5d0-574e2c291c6b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:28.112936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bc4cd-adc8-4730-8479-01b988e3a8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:28.113542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bc4cd-adc8-4730-8479-01b988e3a8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:28.113822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bc4cd-adc8-4730-8479-01b988e3a8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:28.114097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bc4cd-adc8-4730-8479-01b988e3a8c5 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:28.114714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bc4cd-adc8-4730-8479-01b988e3a8c5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:28.115041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bc4cd-adc8-4730-8479-01b988e3a8c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:28.115459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d835cec-2b6d-43bd-9a58-c342fc5a7dab None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:28.116402 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bc4cd-adc8-4730-8479-01b988e3a8c5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:28.122685 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 528/1031] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:28.128342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bc4cd-adc8-4730-8479-01b988e3a8c5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:28.161181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95f591-532e-4aae-9935-c4d4562c1681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:28.162343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95f591-532e-4aae-9935-c4d4562c1681 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:28.162828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95f591-532e-4aae-9935-c4d4562c1681 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:28.163300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95f591-532e-4aae-9935-c4d4562c1681 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:28.166388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95f591-532e-4aae-9935-c4d4562c1681 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:28.167151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95f591-532e-4aae-9935-c4d4562c1681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:28.169647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95f591-532e-4aae-9935-c4d4562c1681 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:28.190018 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 504/1032] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 22 16:54:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:28.191257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95f591-532e-4aae-9935-c4d4562c1681 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:28.246672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bc4cd-adc8-4730-8479-01b988e3a8c5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:28.260933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:28.270965 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:28.286878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507657ab-e6b9-4d76-a493-f06295a3d6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:28.287764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507657ab-e6b9-4d76-a493-f06295a3d6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:28.288191 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507657ab-e6b9-4d76-a493-f06295a3d6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:28.288604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507657ab-e6b9-4d76-a493-f06295a3d6a0 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:28.289371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507657ab-e6b9-4d76-a493-f06295a3d6a0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:28.289815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507657ab-e6b9-4d76-a493-f06295a3d6a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:28.291876 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507657ab-e6b9-4d76-a493-f06295a3d6a0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:28.297280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 529/1033] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:28.303152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507657ab-e6b9-4d76-a493-f06295a3d6a0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:28.309044 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95f591-532e-4aae-9935-c4d4562c1681 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:28.343952 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb1275-086c-4887-9190-82b793db927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:28.344588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb1275-086c-4887-9190-82b793db927d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:28.344869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb1275-086c-4887-9190-82b793db927d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:28.345142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb1275-086c-4887-9190-82b793db927d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:28.356942 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb1275-086c-4887-9190-82b793db927d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:28.358397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb1275-086c-4887-9190-82b793db927d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3fdb1275-086c-4887-9190-82b793db927d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:28.375166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:28.376730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:28.377629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:28.377629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:28.377629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:28.377629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 505/1034] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:28.381870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb1275-086c-4887-9190-82b793db927d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:28.422149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507657ab-e6b9-4d76-a493-f06295a3d6a0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:28.458126 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f49a8-f2a8-4922-877f-4d4595262500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:28.458737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f49a8-f2a8-4922-877f-4d4595262500 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:28.459159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f49a8-f2a8-4922-877f-4d4595262500 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:28.459434 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f49a8-f2a8-4922-877f-4d4595262500 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:28.460127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f49a8-f2a8-4922-877f-4d4595262500 None admin] RBAC: Adding query filter params (interface=public, service_id=3cd840792ae844949b29465009cd5b8b, region_id=RegionOne) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:28.460799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f49a8-f2a8-4922-877f-4d4595262500 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:28.461781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f49a8-f2a8-4922-877f-4d4595262500 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:28.467774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 530/1035] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 22 16:54:28 2022] GET /identity/v3/endpoints?service_id=3cd840792ae844949b29465009cd5b8b&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:28.473548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f49a8-f2a8-4922-877f-4d4595262500 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:28.497521 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb1275-086c-4887-9190-82b793db927d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:28.523004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:28.527249 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:28.534210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd21352-e236-40d5-b3aa-a33ff6f8125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:28.534819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd21352-e236-40d5-b3aa-a33ff6f8125b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:28.535131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd21352-e236-40d5-b3aa-a33ff6f8125b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:28.535408 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd21352-e236-40d5-b3aa-a33ff6f8125b None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:28.536045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd21352-e236-40d5-b3aa-a33ff6f8125b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:28.536346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd21352-e236-40d5-b3aa-a33ff6f8125b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:28.537719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd21352-e236-40d5-b3aa-a33ff6f8125b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:28.554225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 506/1036] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:28.555249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd21352-e236-40d5-b3aa-a33ff6f8125b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:28.601769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f49a8-f2a8-4922-877f-4d4595262500 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:28.639036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37a197-33d2-499a-94be-f7614b5bb567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:28.639648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37a197-33d2-499a-94be-f7614b5bb567 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:28.639933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37a197-33d2-499a-94be-f7614b5bb567 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:28.640322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37a197-33d2-499a-94be-f7614b5bb567 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:28.640932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37a197-33d2-499a-94be-f7614b5bb567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:28.642409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37a197-33d2-499a-94be-f7614b5bb567 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:28.650199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 531/1037] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 22 16:54:28 2022] GET /identity/v3/services => generated 1957 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 16:54:28.705405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd21352-e236-40d5-b3aa-a33ff6f8125b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:28.753297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7625e1-1cd9-4405-88bb-bf661ea72c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:28.754219 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7625e1-1cd9-4405-88bb-bf661ea72c50 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:28.754653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7625e1-1cd9-4405-88bb-bf661ea72c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:28.755115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7625e1-1cd9-4405-88bb-bf661ea72c50 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:28.768821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7625e1-1cd9-4405-88bb-bf661ea72c50 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:28.770309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7625e1-1cd9-4405-88bb-bf661ea72c50 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-dd7625e1-1cd9-4405-88bb-bf661ea72c50 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:54:28.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:28.786189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:28.787173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:28.787173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:28.787173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:54:28.787173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:54:28.787173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:54:28.787173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:54:28.787173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:54:28.787173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:54:28.787173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:54:28.787173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:28.787173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 507/1038] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 22 16:54:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:28.792387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37a197-33d2-499a-94be-f7614b5bb567 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:28.919387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37a197-33d2-499a-94be-f7614b5bb567 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:28.920958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:28.955501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1faa46f-fdd4-46b0-8580-2fa947309644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:28.956145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faa46f-fdd4-46b0-8580-2fa947309644 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:28.956426 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faa46f-fdd4-46b0-8580-2fa947309644 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:28.956701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faa46f-fdd4-46b0-8580-2fa947309644 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:28.957427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1faa46f-fdd4-46b0-8580-2fa947309644 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:28.957729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1faa46f-fdd4-46b0-8580-2fa947309644 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:28.959176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1faa46f-fdd4-46b0-8580-2fa947309644 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:28.978310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 532/1039] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 22 16:54:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:28.984600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7625e1-1cd9-4405-88bb-bf661ea72c50 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:29.036765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:29.099558 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7625e1-1cd9-4405-88bb-bf661ea72c50 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:29.134927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cda856-643b-408a-84be-ff580b135a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:29.135889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cda856-643b-408a-84be-ff580b135a09 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:29.136437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cda856-643b-408a-84be-ff580b135a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:29.136867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cda856-643b-408a-84be-ff580b135a09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:29.211189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cda856-643b-408a-84be-ff580b135a09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:29.213479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cda856-643b-408a-84be-ff580b135a09 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:29.259535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c37b5-92fa-40be-a7c2-1d70d6275890 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:29.259825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c37b5-92fa-40be-a7c2-1d70d6275890 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:29.260106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c37b5-92fa-40be-a7c2-1d70d6275890 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:29.261579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 533/1040] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:54:29.262014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:29.266695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c590f80-e480-43e3-9389-1c27bcca9817 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:29.267007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c590f80-e480-43e3-9389-1c27bcca9817 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:29.267271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c590f80-e480-43e3-9389-1c27bcca9817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:29.287449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1c590f80-e480-43e3-9389-1c27bcca9817 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:29.288555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1c590f80-e480-43e3-9389-1c27bcca9817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:29.328602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 508/1041] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:54:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:54:29.335550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cda856-643b-408a-84be-ff580b135a09 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:29.372471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 534/1042] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:29.439546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cda856-643b-408a-84be-ff580b135a09 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:29.475116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95f378-1196-4dad-a289-91845e49f3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:29.475971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95f378-1196-4dad-a289-91845e49f3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:29.476443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95f378-1196-4dad-a289-91845e49f3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:29.476897 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95f378-1196-4dad-a289-91845e49f3d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:29.487459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95f378-1196-4dad-a289-91845e49f3d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:29.489096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95f378-1196-4dad-a289-91845e49f3d2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:29.491807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-5b95f378-1196-4dad-a289-91845e49f3d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:29.505690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:29.506856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:29.507819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:29.507819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:29.507819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:29.507819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 509/1043] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:29.513397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c590f80-e480-43e3-9389-1c27bcca9817 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:29.607830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:29.619213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c590f80-e480-43e3-9389-1c27bcca9817 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:29.646870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95f378-1196-4dad-a289-91845e49f3d2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:29.653998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763456cf-71d9-4801-949d-7c5674ca2b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:29.654596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763456cf-71d9-4801-949d-7c5674ca2b4b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:29.654872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763456cf-71d9-4801-949d-7c5674ca2b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:29.655205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763456cf-71d9-4801-949d-7c5674ca2b4b None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:29.655808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763456cf-71d9-4801-949d-7c5674ca2b4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:29.656109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763456cf-71d9-4801-949d-7c5674ca2b4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:29.657462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763456cf-71d9-4801-949d-7c5674ca2b4b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:29.669966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 535/1044] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:29.675717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763456cf-71d9-4801-949d-7c5674ca2b4b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:29.760395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95f378-1196-4dad-a289-91845e49f3d2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:29.788142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763456cf-71d9-4801-949d-7c5674ca2b4b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:29.797383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00034a46-4dfd-457a-bf33-9a1c9fe40689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:29.798120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00034a46-4dfd-457a-bf33-9a1c9fe40689 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:29.798292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00034a46-4dfd-457a-bf33-9a1c9fe40689 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:29.798828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00034a46-4dfd-457a-bf33-9a1c9fe40689 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:29.810682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00034a46-4dfd-457a-bf33-9a1c9fe40689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-00034a46-4dfd-457a-bf33-9a1c9fe40689 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:54:29.815493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:29.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:29.818127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 510/1045] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:54:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:29.822878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00034a46-4dfd-457a-bf33-9a1c9fe40689 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:29.827825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504768c-dccd-407d-9f46-f4a3f0844700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:29.827825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504768c-dccd-407d-9f46-f4a3f0844700 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:29.828075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504768c-dccd-407d-9f46-f4a3f0844700 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:29.828305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504768c-dccd-407d-9f46-f4a3f0844700 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:29.835293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504768c-dccd-407d-9f46-f4a3f0844700 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:29.836748 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504768c-dccd-407d-9f46-f4a3f0844700 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-0504768c-dccd-407d-9f46-f4a3f0844700 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:54:29.848308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:29.863840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:29.865135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:54:29.865135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:54:29.865135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:54:29.865135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:54:29.865135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:54:29.865135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:54:29.865135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:29.865135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:29.865135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 536/1046] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:29.865135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504768c-dccd-407d-9f46-f4a3f0844700 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:29.877403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:29.895854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:29.942052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00034a46-4dfd-457a-bf33-9a1c9fe40689 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:29.977194 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504768c-dccd-407d-9f46-f4a3f0844700 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:29.979579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a216f-eb23-4cb6-9d45-62f11c9db554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:29.980740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a216f-eb23-4cb6-9d45-62f11c9db554 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:29.980740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a216f-eb23-4cb6-9d45-62f11c9db554 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:29.980740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a216f-eb23-4cb6-9d45-62f11c9db554 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:29.981345 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a216f-eb23-4cb6-9d45-62f11c9db554 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:29.981643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a216f-eb23-4cb6-9d45-62f11c9db554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:29.983033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a216f-eb23-4cb6-9d45-62f11c9db554 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:29.993037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 511/1047] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:54:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:29.998913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a216f-eb23-4cb6-9d45-62f11c9db554 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:30.013698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd990a-0666-44fc-ab58-c5edd06fb1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:30.014300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd990a-0666-44fc-ab58-c5edd06fb1cf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:30.014572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd990a-0666-44fc-ab58-c5edd06fb1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:30.014844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd990a-0666-44fc-ab58-c5edd06fb1cf None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:30.015570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd990a-0666-44fc-ab58-c5edd06fb1cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:30.015868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd990a-0666-44fc-ab58-c5edd06fb1cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:30.017275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd990a-0666-44fc-ab58-c5edd06fb1cf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:30.030282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 537/1048] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:54:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:30.036550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd990a-0666-44fc-ab58-c5edd06fb1cf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:30.109317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:30.111853 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a216f-eb23-4cb6-9d45-62f11c9db554 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:30.130243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:30.142264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd990a-0666-44fc-ab58-c5edd06fb1cf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:30.152882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088cefb-9162-4af2-8970-61c08df1bfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:30.153974 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088cefb-9162-4af2-8970-61c08df1bfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:30.154386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088cefb-9162-4af2-8970-61c08df1bfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:30.154663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088cefb-9162-4af2-8970-61c08df1bfa8 None admin] PATH_INFO: `/v3/users/e2c0024f8cec4058b6de07cdb22fe4a6` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:30.169596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088cefb-9162-4af2-8970-61c08df1bfa8 None admin] RBAC: Authorizing `identity:get_user(user_id=e2c0024f8cec4058b6de07cdb22fe4a6)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:30.171051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088cefb-9162-4af2-8970-61c08df1bfa8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:30.172830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 512/1049] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:54:29 2022] GET /identity/v3/users/e2c0024f8cec4058b6de07cdb22fe4a6 => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:30.178536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088cefb-9162-4af2-8970-61c08df1bfa8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:30.182332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c0e51-63ed-427e-a351-1d2ea991c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:30.182937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c0e51-63ed-427e-a351-1d2ea991c74b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:30.183234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c0e51-63ed-427e-a351-1d2ea991c74b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:30.183501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c0e51-63ed-427e-a351-1d2ea991c74b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:30.184215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c0e51-63ed-427e-a351-1d2ea991c74b None admin] RBAC: Adding query filter params (role.id=34a4f36a154c4c03942a6a78e370fb70, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=f6c1ce47d8704111988a78e625fd968c) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:30.184516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c0e51-63ed-427e-a351-1d2ea991c74b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:30.185935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c0e51-63ed-427e-a351-1d2ea991c74b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:30.210979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 538/1050] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:54:30 2022] GET /identity/v3/role_assignments?user.id=f6c1ce47d8704111988a78e625fd968c&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=34a4f36a154c4c03942a6a78e370fb70 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:30.293667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088cefb-9162-4af2-8970-61c08df1bfa8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:30.332712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b00db-aa49-48dd-a453-0831b87d7216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:30.333386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b00db-aa49-48dd-a453-0831b87d7216 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:30.333672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b00db-aa49-48dd-a453-0831b87d7216 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:30.333945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b00db-aa49-48dd-a453-0831b87d7216 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:30.386706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:30.411599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b00db-aa49-48dd-a453-0831b87d7216 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:30.413061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b00db-aa49-48dd-a453-0831b87d7216 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:30.519541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 513/1051] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:54:30 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 16:54:30.520893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c0e51-63ed-427e-a351-1d2ea991c74b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:30.622114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c0e51-63ed-427e-a351-1d2ea991c74b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:30.656268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e54af-3f23-45fc-adc2-82be46cd33fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:30.656903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e54af-3f23-45fc-adc2-82be46cd33fa None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:30.657251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e54af-3f23-45fc-adc2-82be46cd33fa None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:30.657590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e54af-3f23-45fc-adc2-82be46cd33fa None admin] PATH_INFO: `/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:30.658768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e54af-3f23-45fc-adc2-82be46cd33fa None admin] RBAC: Authorizing `identity:get_project(project_id=52118eaf3efc4521a61d5f75e3fa7a3a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:30.660286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e54af-3f23-45fc-adc2-82be46cd33fa None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:30.662729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 539/1052] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:54:30 2022] GET /identity/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:30.668170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b00db-aa49-48dd-a453-0831b87d7216 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:30.746045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:30.775645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b00db-aa49-48dd-a453-0831b87d7216 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:30.810889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5a98a-e9c8-47ea-84e2-168d248a0046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:30.811592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5a98a-e9c8-47ea-84e2-168d248a0046 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:30.811872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5a98a-e9c8-47ea-84e2-168d248a0046 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:30.812137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5a98a-e9c8-47ea-84e2-168d248a0046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:30.812851 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5a98a-e9c8-47ea-84e2-168d248a0046 None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=52118eaf3efc4521a61d5f75e3fa7a3a, user.id=e2c0024f8cec4058b6de07cdb22fe4a6) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:30.813153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5a98a-e9c8-47ea-84e2-168d248a0046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:30.814575 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5a98a-e9c8-47ea-84e2-168d248a0046 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:30.827205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 514/1053] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:54:30 2022] GET /identity/v3/role_assignments?user.id=e2c0024f8cec4058b6de07cdb22fe4a6&scope.project.id=52118eaf3efc4521a61d5f75e3fa7a3a&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:30.870262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228938a-f451-468f-8eb9-5d61f4c44e2e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:30.870559 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228938a-f451-468f-8eb9-5d61f4c44e2e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:30.870820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228938a-f451-468f-8eb9-5d61f4c44e2e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:30.872266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 540/1054] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:30.878410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267eba9-34b5-4802-8741-c62be1e51473 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:30.878745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267eba9-34b5-4802-8741-c62be1e51473 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:30.879001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267eba9-34b5-4802-8741-c62be1e51473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:30.898802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a267eba9-34b5-4802-8741-c62be1e51473 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:30.900146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a267eba9-34b5-4802-8741-c62be1e51473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:30.975634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 515/1055] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:31.247097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228938a-f451-468f-8eb9-5d61f4c44e2e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:31.249920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:31.357919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228938a-f451-468f-8eb9-5d61f4c44e2e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:31.392614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011434ce-d6da-4a64-9ac9-6f5672e3d2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:31.393248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011434ce-d6da-4a64-9ac9-6f5672e3d2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:31.393530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011434ce-d6da-4a64-9ac9-6f5672e3d2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:31.393802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011434ce-d6da-4a64-9ac9-6f5672e3d2c6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:31.394376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011434ce-d6da-4a64-9ac9-6f5672e3d2c6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:31.395766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011434ce-d6da-4a64-9ac9-6f5672e3d2c6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-011434ce-d6da-4a64-9ac9-6f5672e3d2c6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:31.405315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:31.406388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 16:54:31.407732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:31.411020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 541/1056] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Sep 22 16:54:31 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:31.416044 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267eba9-34b5-4802-8741-c62be1e51473 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:31.487312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:31.523647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267eba9-34b5-4802-8741-c62be1e51473 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:31.561441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04b115-96b7-4564-b3d1-e10e64527205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:31.562052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04b115-96b7-4564-b3d1-e10e64527205 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:31.562326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04b115-96b7-4564-b3d1-e10e64527205 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:31.562591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04b115-96b7-4564-b3d1-e10e64527205 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:31.563447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04b115-96b7-4564-b3d1-e10e64527205 None admin] RBAC: Adding query filter params (name=image) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:31.563745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04b115-96b7-4564-b3d1-e10e64527205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:31.565109 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04b115-96b7-4564-b3d1-e10e64527205 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:31.570252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 516/1057] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:31.576681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011434ce-d6da-4a64-9ac9-6f5672e3d2c6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:31.645087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:31.680847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011434ce-d6da-4a64-9ac9-6f5672e3d2c6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:31.716474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78444c-b01f-46d1-9f74-0c85d2e40d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:31.717087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78444c-b01f-46d1-9f74-0c85d2e40d46 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:31.717361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78444c-b01f-46d1-9f74-0c85d2e40d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:31.717625 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78444c-b01f-46d1-9f74-0c85d2e40d46 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:31.718254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78444c-b01f-46d1-9f74-0c85d2e40d46 None admin] RBAC: Adding query filter params (type=image) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:31.718548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78444c-b01f-46d1-9f74-0c85d2e40d46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:31.719951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78444c-b01f-46d1-9f74-0c85d2e40d46 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:31.730584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 542/1058] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:31.731894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04b115-96b7-4564-b3d1-e10e64527205 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:31.831998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04b115-96b7-4564-b3d1-e10e64527205 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:31.865108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5d8ab-c8bd-4dba-a1f9-2945ddf9aa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:31.865715 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5d8ab-c8bd-4dba-a1f9-2945ddf9aa49 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:31.865991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5d8ab-c8bd-4dba-a1f9-2945ddf9aa49 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:31.866260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5d8ab-c8bd-4dba-a1f9-2945ddf9aa49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:31.866925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5d8ab-c8bd-4dba-a1f9-2945ddf9aa49 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:31.869000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5d8ab-c8bd-4dba-a1f9-2945ddf9aa49 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:31.879734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:31.907288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 517/1059] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 16:54:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:32.366559 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ab6db-49be-400c-9ace-a3a8549ad7cf None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:32.366882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ab6db-49be-400c-9ace-a3a8549ad7cf None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:32.367008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ab6db-49be-400c-9ace-a3a8549ad7cf None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:32.368803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 543/1060] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:32.374495 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358969f-48d9-4da9-85b0-389d8f760e6f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:32.374819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358969f-48d9-4da9-85b0-389d8f760e6f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:32.375087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358969f-48d9-4da9-85b0-389d8f760e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:32.393887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5358969f-48d9-4da9-85b0-389d8f760e6f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:32.395104 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5358969f-48d9-4da9-85b0-389d8f760e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:32.476591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 518/1061] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:32.739972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ab6db-49be-400c-9ace-a3a8549ad7cf None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:32.857831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ab6db-49be-400c-9ace-a3a8549ad7cf None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:32.890359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:32.898141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f13a3-f50f-480e-8394-e5199cba5dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:32.898796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f13a3-f50f-480e-8394-e5199cba5dab None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:32.899103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f13a3-f50f-480e-8394-e5199cba5dab None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:32.899410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f13a3-f50f-480e-8394-e5199cba5dab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:32.911405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f13a3-f50f-480e-8394-e5199cba5dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-d22f13a3-f50f-480e-8394-e5199cba5dab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:54:32.916888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:32.917890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:32.919943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 544/1062] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 22 16:54:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:32.924358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358969f-48d9-4da9-85b0-389d8f760e6f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:33.006695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61876c-c5a0-46ec-a203-b416350e8d43 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:33.006888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61876c-c5a0-46ec-a203-b416350e8d43 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:33.007201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61876c-c5a0-46ec-a203-b416350e8d43 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:33.009160 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 545/1063] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:54:33.014323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d0686-3a3b-41be-a797-18c278210549 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:33.014770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d0686-3a3b-41be-a797-18c278210549 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:33.015012 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d0686-3a3b-41be-a797-18c278210549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:33.032063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358969f-48d9-4da9-85b0-389d8f760e6f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:33.035668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-df5d0686-3a3b-41be-a797-18c278210549 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:33.036841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-df5d0686-3a3b-41be-a797-18c278210549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:33.051924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:33.075015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61cb89-e50d-418e-add5-d7e1fad717eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:33.075589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61cb89-e50d-418e-add5-d7e1fad717eb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:33.075865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61cb89-e50d-418e-add5-d7e1fad717eb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:33.076133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61cb89-e50d-418e-add5-d7e1fad717eb None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:33.076788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61cb89-e50d-418e-add5-d7e1fad717eb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:33.077090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61cb89-e50d-418e-add5-d7e1fad717eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:33.078457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61cb89-e50d-418e-add5-d7e1fad717eb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:33.090984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 519/1064] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 22 16:54:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:33.092607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61cb89-e50d-418e-add5-d7e1fad717eb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:33.118104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 546/1065] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:33.207588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61cb89-e50d-418e-add5-d7e1fad717eb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:33.244603 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb877cf6-fc50-45af-bcd8-ea2cc76b4b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:33.245572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb877cf6-fc50-45af-bcd8-ea2cc76b4b48 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:33.246025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb877cf6-fc50-45af-bcd8-ea2cc76b4b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:33.246475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb877cf6-fc50-45af-bcd8-ea2cc76b4b48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:33.257986 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb877cf6-fc50-45af-bcd8-ea2cc76b4b48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:33.259783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb877cf6-fc50-45af-bcd8-ea2cc76b4b48 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-bb877cf6-fc50-45af-bcd8-ea2cc76b4b48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:33.275976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:33.276971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:33.277880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:33.277880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:33.277880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:33.279686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 520/1066] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:33.285535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d0686-3a3b-41be-a797-18c278210549 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:33.395168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb877cf6-fc50-45af-bcd8-ea2cc76b4b48 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:33.398979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d0686-3a3b-41be-a797-18c278210549 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:33.420858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:33.429634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:33.438393 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636f385-63bd-40c8-94ea-c6694fb8e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:33.439034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636f385-63bd-40c8-94ea-c6694fb8e87c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:33.439318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636f385-63bd-40c8-94ea-c6694fb8e87c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:33.439598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636f385-63bd-40c8-94ea-c6694fb8e87c None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:33.440243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636f385-63bd-40c8-94ea-c6694fb8e87c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:33.440543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636f385-63bd-40c8-94ea-c6694fb8e87c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:33.442020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636f385-63bd-40c8-94ea-c6694fb8e87c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:33.454121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 547/1067] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:33.459526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636f385-63bd-40c8-94ea-c6694fb8e87c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:33.511143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb877cf6-fc50-45af-bcd8-ea2cc76b4b48 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:33.548450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf090b-655e-4375-a207-a3bf16bfe1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:33.549334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf090b-655e-4375-a207-a3bf16bfe1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:33.549620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf090b-655e-4375-a207-a3bf16bfe1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:33.549888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf090b-655e-4375-a207-a3bf16bfe1f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:33.561567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf090b-655e-4375-a207-a3bf16bfe1f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-43cf090b-655e-4375-a207-a3bf16bfe1f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:54:33.566538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:33.567516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:54:33.568486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:33.569668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636f385-63bd-40c8-94ea-c6694fb8e87c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:33.569759 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 521/1068] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:54:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:33.573638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf090b-655e-4375-a207-a3bf16bfe1f9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:33.607881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89a9f1-802b-4152-90a9-55df0023b3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:33.608532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a9f1-802b-4152-90a9-55df0023b3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:33.608818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a9f1-802b-4152-90a9-55df0023b3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:33.609092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a9f1-802b-4152-90a9-55df0023b3e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:33.619779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89a9f1-802b-4152-90a9-55df0023b3e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:33.621304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89a9f1-802b-4152-90a9-55df0023b3e8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-9c89a9f1-802b-4152-90a9-55df0023b3e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:54:33.634124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:33.635330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:33.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:33.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:33.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:54:33.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:54:33.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:54:33.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:54:33.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:54:33.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:54:33.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:54:33.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:33.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 548/1069] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 22 16:54:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:33.642245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89a9f1-802b-4152-90a9-55df0023b3e8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:33.679772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:33.689833 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:33.692802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf090b-655e-4375-a207-a3bf16bfe1f9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:33.727027 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977edcb-1057-45db-814e-b0636febd029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:33.728161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977edcb-1057-45db-814e-b0636febd029 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:33.728435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977edcb-1057-45db-814e-b0636febd029 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:33.728699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977edcb-1057-45db-814e-b0636febd029 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:33.729347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977edcb-1057-45db-814e-b0636febd029 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:33.729640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977edcb-1057-45db-814e-b0636febd029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:33.731015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977edcb-1057-45db-814e-b0636febd029 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:33.739800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 522/1070] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:54:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:33.746066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977edcb-1057-45db-814e-b0636febd029 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:33.765188 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89a9f1-802b-4152-90a9-55df0023b3e8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:33.803910 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de1384-dfdb-45a9-a68d-5c2c9bc37bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:33.804518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1384-dfdb-45a9-a68d-5c2c9bc37bfb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:33.804805 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1384-dfdb-45a9-a68d-5c2c9bc37bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:33.805079 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1384-dfdb-45a9-a68d-5c2c9bc37bfb None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:33.805914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de1384-dfdb-45a9-a68d-5c2c9bc37bfb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:33.806223 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de1384-dfdb-45a9-a68d-5c2c9bc37bfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:33.807674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de1384-dfdb-45a9-a68d-5c2c9bc37bfb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:33.826160 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 549/1071] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 22 16:54:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:33.831796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de1384-dfdb-45a9-a68d-5c2c9bc37bfb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:33.857705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977edcb-1057-45db-814e-b0636febd029 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:33.895602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217ab66-53a6-441a-9e9b-e5193b08e85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:33.896234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217ab66-53a6-441a-9e9b-e5193b08e85f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:33.896513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217ab66-53a6-441a-9e9b-e5193b08e85f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:33.896783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217ab66-53a6-441a-9e9b-e5193b08e85f None admin] PATH_INFO: `/v3/users/e2c0024f8cec4058b6de07cdb22fe4a6` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:33.897906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217ab66-53a6-441a-9e9b-e5193b08e85f None admin] RBAC: Authorizing `identity:get_user(user_id=e2c0024f8cec4058b6de07cdb22fe4a6)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:33.899387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217ab66-53a6-441a-9e9b-e5193b08e85f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:33.900977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 523/1072] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:54:33 2022] GET /identity/v3/users/e2c0024f8cec4058b6de07cdb22fe4a6 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:33.906613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217ab66-53a6-441a-9e9b-e5193b08e85f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:33.929521 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:33.941288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:33.946991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de1384-dfdb-45a9-a68d-5c2c9bc37bfb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:33.983817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d4c53-ee6a-41f3-9b61-279b73a58a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:33.984454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d4c53-ee6a-41f3-9b61-279b73a58a3f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:33.984727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d4c53-ee6a-41f3-9b61-279b73a58a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:33.984990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d4c53-ee6a-41f3-9b61-279b73a58a3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:33.997271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d4c53-ee6a-41f3-9b61-279b73a58a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:33.998795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d4c53-ee6a-41f3-9b61-279b73a58a3f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e26d4c53-ee6a-41f3-9b61-279b73a58a3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:34.015992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:34.017009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:34.023353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:34.023353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:34.023353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:34.023555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217ab66-53a6-441a-9e9b-e5193b08e85f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:34.023645 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 550/1073] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:34.023645 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d4c53-ee6a-41f3-9b61-279b73a58a3f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:34.059602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de63c98-83fb-4fff-a437-80c0c6ffa1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:34.060231 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de63c98-83fb-4fff-a437-80c0c6ffa1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:34.060506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de63c98-83fb-4fff-a437-80c0c6ffa1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:34.060770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de63c98-83fb-4fff-a437-80c0c6ffa1c1 None admin] PATH_INFO: `/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:34.061888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de63c98-83fb-4fff-a437-80c0c6ffa1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=52118eaf3efc4521a61d5f75e3fa7a3a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:34.063419 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de63c98-83fb-4fff-a437-80c0c6ffa1c1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:34.065244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 524/1074] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:54:33 2022] GET /identity/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:34.070846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de63c98-83fb-4fff-a437-80c0c6ffa1c1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:34.133353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d4c53-ee6a-41f3-9b61-279b73a58a3f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:34.160250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:34.168598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e1c24-e400-47b0-a7ca-1b842c631cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:34.169199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e1c24-e400-47b0-a7ca-1b842c631cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:34.169477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e1c24-e400-47b0-a7ca-1b842c631cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:34.169746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e1c24-e400-47b0-a7ca-1b842c631cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:34.170374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e1c24-e400-47b0-a7ca-1b842c631cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:34.170670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e1c24-e400-47b0-a7ca-1b842c631cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:34.172048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e1c24-e400-47b0-a7ca-1b842c631cf5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:34.173426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de63c98-83fb-4fff-a437-80c0c6ffa1c1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:34.184732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 551/1075] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:34.186696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcfa7d-f646-4e47-90ae-1fbe1105c02d None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:34.187013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcfa7d-f646-4e47-90ae-1fbe1105c02d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:34.187351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcfa7d-f646-4e47-90ae-1fbe1105c02d None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:34.189411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 552/1076] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:34.190398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcfa7d-f646-4e47-90ae-1fbe1105c02d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:34.203707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:34.210556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b853b3-d754-4465-ad97-da15574cee19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:34.211261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b853b3-d754-4465-ad97-da15574cee19 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:34.211548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b853b3-d754-4465-ad97-da15574cee19 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:34.211854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b853b3-d754-4465-ad97-da15574cee19 None admin] PATH_INFO: `/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a/users/e2c0024f8cec4058b6de07cdb22fe4a6/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:34.213767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b853b3-d754-4465-ad97-da15574cee19 None admin] RBAC: Authorizing `identity:create_grant(project_id=52118eaf3efc4521a61d5f75e3fa7a3a, user_id=e2c0024f8cec4058b6de07cdb22fe4a6, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:34.215262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b853b3-d754-4465-ad97-da15574cee19 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:34.228810 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 525/1077] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:54:34 2022] PUT /identity/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a/users/e2c0024f8cec4058b6de07cdb22fe4a6/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:54:34.230113 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda1f64-67d4-41eb-acd7-f77f7458bb05 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:34.230405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda1f64-67d4-41eb-acd7-f77f7458bb05 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:34.230682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda1f64-67d4-41eb-acd7-f77f7458bb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:34.252066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ecda1f64-67d4-41eb-acd7-f77f7458bb05 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:34.253309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ecda1f64-67d4-41eb-acd7-f77f7458bb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:34.304774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcfa7d-f646-4e47-90ae-1fbe1105c02d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:34.340231 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 526/1078] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:34.348750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955427f-bf8f-43dd-9cc5-50ebb91ee2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:34.349794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955427f-bf8f-43dd-9cc5-50ebb91ee2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:34.350270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955427f-bf8f-43dd-9cc5-50ebb91ee2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:34.350835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955427f-bf8f-43dd-9cc5-50ebb91ee2f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:34.359559 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955427f-bf8f-43dd-9cc5-50ebb91ee2f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:34.361291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955427f-bf8f-43dd-9cc5-50ebb91ee2f8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a955427f-bf8f-43dd-9cc5-50ebb91ee2f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:54:34.375131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:34.376318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:34.377448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:54:34.377448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:54:34.377448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:54:34.377448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:54:34.377448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:54:34.377448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:54:34.377448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:34.377448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:34.384755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 553/1079] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:34.386273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda1f64-67d4-41eb-acd7-f77f7458bb05 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:34.471543 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:34.496646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda1f64-67d4-41eb-acd7-f77f7458bb05 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:34.532433 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12336cd-b2ee-4ff3-a129-d868dc2d45df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:34.533040 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12336cd-b2ee-4ff3-a129-d868dc2d45df None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:34.533322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12336cd-b2ee-4ff3-a129-d868dc2d45df None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:34.533593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12336cd-b2ee-4ff3-a129-d868dc2d45df None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:34.534285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12336cd-b2ee-4ff3-a129-d868dc2d45df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:34.534592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12336cd-b2ee-4ff3-a129-d868dc2d45df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:34.536061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12336cd-b2ee-4ff3-a129-d868dc2d45df None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:34.553562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 527/1080] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:54:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:34.555166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955427f-bf8f-43dd-9cc5-50ebb91ee2f8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:34.611753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12336cd-b2ee-4ff3-a129-d868dc2d45df None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:34.631514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:34.664814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955427f-bf8f-43dd-9cc5-50ebb91ee2f8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:34.700997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b775d-5ee0-42b7-8403-12967020bc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:34.701661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b775d-5ee0-42b7-8403-12967020bc4d None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:34.701939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b775d-5ee0-42b7-8403-12967020bc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:34.702211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b775d-5ee0-42b7-8403-12967020bc4d None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/f6c1ce47d8704111988a78e625fd968c/roles/34a4f36a154c4c03942a6a78e370fb70` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:34.704406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b775d-5ee0-42b7-8403-12967020bc4d None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=f6c1ce47d8704111988a78e625fd968c, role_id=34a4f36a154c4c03942a6a78e370fb70)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:34.705864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b775d-5ee0-42b7-8403-12967020bc4d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:34.720192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 554/1081] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:54:34 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/f6c1ce47d8704111988a78e625fd968c/roles/34a4f36a154c4c03942a6a78e370fb70 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:54:34.745349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12336cd-b2ee-4ff3-a129-d868dc2d45df None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:34.795205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b859c-fc47-4c81-aa87-44bc061b90a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:34.795834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b859c-fc47-4c81-aa87-44bc061b90a1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:34.796110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b859c-fc47-4c81-aa87-44bc061b90a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:34.796382 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b859c-fc47-4c81-aa87-44bc061b90a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:34.807201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:34.874895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b859c-fc47-4c81-aa87-44bc061b90a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:34.876410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b859c-fc47-4c81-aa87-44bc061b90a1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:34.986740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 528/1082] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:54:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 16:54:34.992567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b775d-5ee0-42b7-8403-12967020bc4d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:35.095319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b775d-5ee0-42b7-8403-12967020bc4d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:35.126373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:35.134138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210e6c2-c7e8-4d32-a13f-1fa133776c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:35.135197 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210e6c2-c7e8-4d32-a13f-1fa133776c44 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:35.135937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210e6c2-c7e8-4d32-a13f-1fa133776c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:35.136671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210e6c2-c7e8-4d32-a13f-1fa133776c44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:35.147776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210e6c2-c7e8-4d32-a13f-1fa133776c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:35.149250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210e6c2-c7e8-4d32-a13f-1fa133776c44 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-4210e6c2-c7e8-4d32-a13f-1fa133776c44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:35.163320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:35.164317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:35.165742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:35.165742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:35.165742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:35.169919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 555/1083] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:35.171316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b859c-fc47-4c81-aa87-44bc061b90a1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:35.211614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:35.273223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b859c-fc47-4c81-aa87-44bc061b90a1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:35.309143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d88148-1c88-4921-a85b-5bbb5321c3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:35.309693 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d88148-1c88-4921-a85b-5bbb5321c3ac None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:35.309972 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d88148-1c88-4921-a85b-5bbb5321c3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:35.310245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d88148-1c88-4921-a85b-5bbb5321c3ac None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:35.310888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d88148-1c88-4921-a85b-5bbb5321c3ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:35.311368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d88148-1c88-4921-a85b-5bbb5321c3ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:35.312739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d88148-1c88-4921-a85b-5bbb5321c3ac None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:35.324809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 529/1084] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:36.418883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef9fb0-23e8-43e7-8c59-b63867f5b380 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:36.419457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef9fb0-23e8-43e7-8c59-b63867f5b380 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:36.419624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef9fb0-23e8-43e7-8c59-b63867f5b380 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:36.421031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 556/1085] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:36.426745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086711b-2336-4648-934a-150c30eb9fbe None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:36.427343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086711b-2336-4648-934a-150c30eb9fbe None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:36.427343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086711b-2336-4648-934a-150c30eb9fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:36.446051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9086711b-2336-4648-934a-150c30eb9fbe None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:36.447257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9086711b-2336-4648-934a-150c30eb9fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:36.513400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 530/1086] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:36.783278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef9fb0-23e8-43e7-8c59-b63867f5b380 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:36.893027 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef9fb0-23e8-43e7-8c59-b63867f5b380 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:36.928124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018ebb8-235c-48cf-b6cb-c8f67db824bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:36.928751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ebb8-235c-48cf-b6cb-c8f67db824bf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:36.929032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ebb8-235c-48cf-b6cb-c8f67db824bf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:36.929299 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ebb8-235c-48cf-b6cb-c8f67db824bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:36.938800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c72de0-606e-4ef3-ae58-62b9233e9b60 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:36.939121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c72de0-606e-4ef3-ae58-62b9233e9b60 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:36.939399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c72de0-606e-4ef3-ae58-62b9233e9b60 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:36.939964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018ebb8-235c-48cf-b6cb-c8f67db824bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:54:36.940772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 531/1087] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-4018ebb8-235c-48cf-b6cb-c8f67db824bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:54:36.944971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:36.946058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:54:36.947000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:36.947660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486e56b-d3c4-4b2a-9ebb-49d02f5999bc None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:36.947740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 557/1088] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:54:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:36.947809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486e56b-d3c4-4b2a-9ebb-49d02f5999bc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:36.947809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486e56b-d3c4-4b2a-9ebb-49d02f5999bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:36.951944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018ebb8-235c-48cf-b6cb-c8f67db824bf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:36.965234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6486e56b-d3c4-4b2a-9ebb-49d02f5999bc None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:36.966423 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6486e56b-d3c4-4b2a-9ebb-49d02f5999bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:36.989524 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:36.994265 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:37.034387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 532/1089] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:54:37.053582 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018ebb8-235c-48cf-b6cb-c8f67db824bf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:37.086426 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9f6a0-2bb6-4316-8ac6-c1f377de1290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:37.087531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9f6a0-2bb6-4316-8ac6-c1f377de1290 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:37.087804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9f6a0-2bb6-4316-8ac6-c1f377de1290 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:37.088071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9f6a0-2bb6-4316-8ac6-c1f377de1290 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:37.088713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9f6a0-2bb6-4316-8ac6-c1f377de1290 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:37.089002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9f6a0-2bb6-4316-8ac6-c1f377de1290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:37.090344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9f6a0-2bb6-4316-8ac6-c1f377de1290 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:37.098965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 558/1090] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:54:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:37.105340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486e56b-d3c4-4b2a-9ebb-49d02f5999bc None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:37.201678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486e56b-d3c4-4b2a-9ebb-49d02f5999bc None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:37.234698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81cd34-ca7a-43f9-a489-e952dd9c1c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:37.235354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81cd34-ca7a-43f9-a489-e952dd9c1c09 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:37.235628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81cd34-ca7a-43f9-a489-e952dd9c1c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:37.235895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81cd34-ca7a-43f9-a489-e952dd9c1c09 None admin] PATH_INFO: `/v3/users/e2c0024f8cec4058b6de07cdb22fe4a6` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:37.237308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81cd34-ca7a-43f9-a489-e952dd9c1c09 None admin] RBAC: Authorizing `identity:get_user(user_id=e2c0024f8cec4058b6de07cdb22fe4a6)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:37.238741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81cd34-ca7a-43f9-a489-e952dd9c1c09 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:37.240348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 533/1091] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:54:37 2022] GET /identity/v3/users/e2c0024f8cec4058b6de07cdb22fe4a6 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:37.246674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9f6a0-2bb6-4316-8ac6-c1f377de1290 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:37.303680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81cd34-ca7a-43f9-a489-e952dd9c1c09 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:37.331146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:37.346848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9f6a0-2bb6-4316-8ac6-c1f377de1290 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:37.349193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:37.395437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f41f2-11c2-441b-9b46-2272b062b278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:37.396060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f41f2-11c2-441b-9b46-2272b062b278 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:37.396335 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f41f2-11c2-441b-9b46-2272b062b278 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:37.396603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f41f2-11c2-441b-9b46-2272b062b278 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:37.421228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81cd34-ca7a-43f9-a489-e952dd9c1c09 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:37.457613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eab95a-9a31-4061-88fc-1405a3ee9cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:37.458667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eab95a-9a31-4061-88fc-1405a3ee9cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:37.459051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eab95a-9a31-4061-88fc-1405a3ee9cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:37.459328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eab95a-9a31-4061-88fc-1405a3ee9cd1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:37.471094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eab95a-9a31-4061-88fc-1405a3ee9cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:54:37.473504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f41f2-11c2-441b-9b46-2272b062b278 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:37.475476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f41f2-11c2-441b-9b46-2272b062b278 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-e5eab95a-9a31-4061-88fc-1405a3ee9cd1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:54:37.476077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:37.477005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:37.478004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 534/1092] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 22 16:54:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:37.484657 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eab95a-9a31-4061-88fc-1405a3ee9cd1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:37.578742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:37.583307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 559/1093] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:54:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:54:37.590876 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f41f2-11c2-441b-9b46-2272b062b278 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:37.594448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:37.598916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eab95a-9a31-4061-88fc-1405a3ee9cd1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:37.634414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4587e0d-fecf-468d-8007-605621a71cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:37.635033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4587e0d-fecf-468d-8007-605621a71cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:37.635318 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4587e0d-fecf-468d-8007-605621a71cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:37.635583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4587e0d-fecf-468d-8007-605621a71cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:37.636230 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4587e0d-fecf-468d-8007-605621a71cf6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:37.636524 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4587e0d-fecf-468d-8007-605621a71cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:37.637882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4587e0d-fecf-468d-8007-605621a71cf6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:37.645907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 535/1094] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 22 16:54:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:37.651790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4587e0d-fecf-468d-8007-605621a71cf6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:37.695322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f41f2-11c2-441b-9b46-2272b062b278 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:37.730586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db961052-d321-4f9d-885e-7a8e30fb67a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:37.731229 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db961052-d321-4f9d-885e-7a8e30fb67a5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:37.731500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db961052-d321-4f9d-885e-7a8e30fb67a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:37.731767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db961052-d321-4f9d-885e-7a8e30fb67a5 None admin] PATH_INFO: `/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:37.732870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db961052-d321-4f9d-885e-7a8e30fb67a5 None admin] RBAC: Authorizing `identity:get_project(project_id=52118eaf3efc4521a61d5f75e3fa7a3a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:37.734282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db961052-d321-4f9d-885e-7a8e30fb67a5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:37.735918 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 560/1095] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:54:37 2022] GET /identity/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:37.742048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db961052-d321-4f9d-885e-7a8e30fb67a5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:37.757405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4587e0d-fecf-468d-8007-605621a71cf6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:37.792326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffce895-dd98-4014-b64e-acfb423d2532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:37.792959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffce895-dd98-4014-b64e-acfb423d2532 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:37.793241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffce895-dd98-4014-b64e-acfb423d2532 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:37.793515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffce895-dd98-4014-b64e-acfb423d2532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:37.804795 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffce895-dd98-4014-b64e-acfb423d2532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:37.806805 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffce895-dd98-4014-b64e-acfb423d2532 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:37.814929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:37.815081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-8ffce895-dd98-4014-b64e-acfb423d2532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:37.821994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:37.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:37.823984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:37.823984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:37.823984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:37.823984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 536/1096] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:37.828830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffce895-dd98-4014-b64e-acfb423d2532 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:37.845096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db961052-d321-4f9d-885e-7a8e30fb67a5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:37.878667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98781e30-4685-4fc3-80e6-3324f7e20d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:37.879318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98781e30-4685-4fc3-80e6-3324f7e20d24 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:37.879599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98781e30-4685-4fc3-80e6-3324f7e20d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:37.879875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98781e30-4685-4fc3-80e6-3324f7e20d24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:37.880587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98781e30-4685-4fc3-80e6-3324f7e20d24 None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=52118eaf3efc4521a61d5f75e3fa7a3a, user.id=e2c0024f8cec4058b6de07cdb22fe4a6) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:37.880899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98781e30-4685-4fc3-80e6-3324f7e20d24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:37.882308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98781e30-4685-4fc3-80e6-3324f7e20d24 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:37.888145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 561/1097] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:54:37 2022] GET /identity/v3/role_assignments?user.id=e2c0024f8cec4058b6de07cdb22fe4a6&scope.project.id=52118eaf3efc4521a61d5f75e3fa7a3a&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:37.941952 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffce895-dd98-4014-b64e-acfb423d2532 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:37.979022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01a4cc-4914-4c03-b41e-6510a2b6d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:37.979644 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01a4cc-4914-4c03-b41e-6510a2b6d4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:37.979926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01a4cc-4914-4c03-b41e-6510a2b6d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:37.980200 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01a4cc-4914-4c03-b41e-6510a2b6d4b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:37.980952 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01a4cc-4914-4c03-b41e-6510a2b6d4b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:37.981168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01a4cc-4914-4c03-b41e-6510a2b6d4b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:37.982590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01a4cc-4914-4c03-b41e-6510a2b6d4b2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:37.996061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 537/1098] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:38.001867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98781e30-4685-4fc3-80e6-3324f7e20d24 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:38.105588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98781e30-4685-4fc3-80e6-3324f7e20d24 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:38.138425 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c187c-9577-4038-bdd6-6a31726e23af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:38.139066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c187c-9577-4038-bdd6-6a31726e23af None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:38.139344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c187c-9577-4038-bdd6-6a31726e23af None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:38.139617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c187c-9577-4038-bdd6-6a31726e23af None admin] PATH_INFO: `/v3/users/nova` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:38.142243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:38.147575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c187c-9577-4038-bdd6-6a31726e23af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:38.149133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c187c-9577-4038-bdd6-6a31726e23af None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-fa2c187c-9577-4038-bdd6-6a31726e23af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:54:38.160279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:38.161378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:38.162328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:38.162328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:38.162328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:54:38.162328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:54:38.162328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:54:38.162328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:54:38.162328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:54:38.162328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:54:38.162328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 16:54:38.162328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:38.162328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 562/1099] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 22 16:54:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:54:38.167694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01a4cc-4914-4c03-b41e-6510a2b6d4b2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:38.227601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:38.271542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01a4cc-4914-4c03-b41e-6510a2b6d4b2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:38.305088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065cccc-37e7-48df-82ce-b31b043b02af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:38.306193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065cccc-37e7-48df-82ce-b31b043b02af None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:38.306474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065cccc-37e7-48df-82ce-b31b043b02af None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:38.306744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065cccc-37e7-48df-82ce-b31b043b02af None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:38.307498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065cccc-37e7-48df-82ce-b31b043b02af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:38.307801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065cccc-37e7-48df-82ce-b31b043b02af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:38.309222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065cccc-37e7-48df-82ce-b31b043b02af None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:38.326876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 538/1100] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 22 16:54:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:38.332586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c187c-9577-4038-bdd6-6a31726e23af None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:38.434158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c187c-9577-4038-bdd6-6a31726e23af None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:38.468652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad6b3e-be9d-424b-9ec0-96f5f837696f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:38.469253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad6b3e-be9d-424b-9ec0-96f5f837696f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:38.469533 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad6b3e-be9d-424b-9ec0-96f5f837696f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:38.469802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad6b3e-be9d-424b-9ec0-96f5f837696f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:38.534907 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:38.542334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad6b3e-be9d-424b-9ec0-96f5f837696f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:38.543814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad6b3e-be9d-424b-9ec0-96f5f837696f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:38.645863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 563/1101] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:54:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 16:54:38.646840 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065cccc-37e7-48df-82ce-b31b043b02af None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:38.744139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065cccc-37e7-48df-82ce-b31b043b02af None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:38.762785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:38.780098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32097078-49a3-4b68-8a52-83ddb85be0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:38.780736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32097078-49a3-4b68-8a52-83ddb85be0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:38.781020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32097078-49a3-4b68-8a52-83ddb85be0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:38.781296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32097078-49a3-4b68-8a52-83ddb85be0d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:38.793100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32097078-49a3-4b68-8a52-83ddb85be0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:38.794585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32097078-49a3-4b68-8a52-83ddb85be0d7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-32097078-49a3-4b68-8a52-83ddb85be0d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:54:38.811001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:38.812029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:54:38.813361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:54:38.813361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:54:38.813361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:38.813361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 539/1102] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:38.817873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad6b3e-be9d-424b-9ec0-96f5f837696f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:38.907007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:38.912943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad6b3e-be9d-424b-9ec0-96f5f837696f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:38.946960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97276776-ed27-4a8e-b402-dd98a99e30ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:38.947626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97276776-ed27-4a8e-b402-dd98a99e30ec None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:38.947960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97276776-ed27-4a8e-b402-dd98a99e30ec None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:38.948234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97276776-ed27-4a8e-b402-dd98a99e30ec None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:38.948860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97276776-ed27-4a8e-b402-dd98a99e30ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:38.949158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97276776-ed27-4a8e-b402-dd98a99e30ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:38.950496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97276776-ed27-4a8e-b402-dd98a99e30ec None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:38.961728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 564/1103] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:38.967381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32097078-49a3-4b68-8a52-83ddb85be0d7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:39.065254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32097078-49a3-4b68-8a52-83ddb85be0d7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:39.097892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0463b1-8d88-42db-9557-6fba1d9ee61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:39.098509 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0463b1-8d88-42db-9557-6fba1d9ee61d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:39.098787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0463b1-8d88-42db-9557-6fba1d9ee61d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:39.099087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0463b1-8d88-42db-9557-6fba1d9ee61d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:39.106136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0463b1-8d88-42db-9557-6fba1d9ee61d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:39.107601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0463b1-8d88-42db-9557-6fba1d9ee61d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-4d0463b1-8d88-42db-9557-6fba1d9ee61d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:54:39.119462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:39.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:39.121377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:54:39.121377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:54:39.121377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:54:39.121377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:54:39.121377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:54:39.121377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:54:39.121377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:54:39.121377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:54:39.121377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 540/1104] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:54:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:39.126456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97276776-ed27-4a8e-b402-dd98a99e30ec None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:39.224207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97276776-ed27-4a8e-b402-dd98a99e30ec None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:39.258225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211671a-b00e-47bd-9588-e9a7c83df474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:39.258856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211671a-b00e-47bd-9588-e9a7c83df474 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:39.259152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211671a-b00e-47bd-9588-e9a7c83df474 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:39.259421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211671a-b00e-47bd-9588-e9a7c83df474 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:39.260096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211671a-b00e-47bd-9588-e9a7c83df474 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:39.260395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211671a-b00e-47bd-9588-e9a7c83df474 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:39.261799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211671a-b00e-47bd-9588-e9a7c83df474 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:39.273550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 565/1105] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:54:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:39.280366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0463b1-8d88-42db-9557-6fba1d9ee61d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:39.311247 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:39.379415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0463b1-8d88-42db-9557-6fba1d9ee61d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:39.412300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed78947-a395-4029-8d83-a7477e2c1cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:39.412895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed78947-a395-4029-8d83-a7477e2c1cfc None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:39.413175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed78947-a395-4029-8d83-a7477e2c1cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:39.413447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed78947-a395-4029-8d83-a7477e2c1cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:39.414173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed78947-a395-4029-8d83-a7477e2c1cfc None admin] RBAC: Adding query filter params (role.id=34a4f36a154c4c03942a6a78e370fb70, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=f6c1ce47d8704111988a78e625fd968c) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:39.414480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed78947-a395-4029-8d83-a7477e2c1cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:39.417187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed78947-a395-4029-8d83-a7477e2c1cfc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:39.424948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 541/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:54:39 2022] GET /identity/v3/role_assignments?user.id=f6c1ce47d8704111988a78e625fd968c&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=34a4f36a154c4c03942a6a78e370fb70 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:40.079217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b40b3-71c9-4242-9902-dad753407921 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:40.079709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b40b3-71c9-4242-9902-dad753407921 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:40.079709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b40b3-71c9-4242-9902-dad753407921 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:40.081382 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 566/1107] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:40.087369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcabc31-bb34-4700-9813-6ddc4eebcd8c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:40.088017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcabc31-bb34-4700-9813-6ddc4eebcd8c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:40.088017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcabc31-bb34-4700-9813-6ddc4eebcd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:40.106998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-cfcabc31-bb34-4700-9813-6ddc4eebcd8c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:40.108198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cfcabc31-bb34-4700-9813-6ddc4eebcd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:40.176817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 542/1108] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:40.448716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b40b3-71c9-4242-9902-dad753407921 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:40.467401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:40.552875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b40b3-71c9-4242-9902-dad753407921 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:40.588014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f1d2f-313f-45f0-a74a-600d3045dc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:40.588636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f1d2f-313f-45f0-a74a-600d3045dc91 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:40.588917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f1d2f-313f-45f0-a74a-600d3045dc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:40.589188 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f1d2f-313f-45f0-a74a-600d3045dc91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:40.590280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f1d2f-313f-45f0-a74a-600d3045dc91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:40.591720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f1d2f-313f-45f0-a74a-600d3045dc91 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:40.593287 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 567/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:40.599141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcabc31-bb34-4700-9813-6ddc4eebcd8c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:40.623535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:40.695775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcabc31-bb34-4700-9813-6ddc4eebcd8c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:40.728461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffab43-a22f-4184-8bc0-8ccc297568fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:40.728988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffab43-a22f-4184-8bc0-8ccc297568fd None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:40.729262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffab43-a22f-4184-8bc0-8ccc297568fd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:40.729531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffab43-a22f-4184-8bc0-8ccc297568fd None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:40.730176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffab43-a22f-4184-8bc0-8ccc297568fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:40.731613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffab43-a22f-4184-8bc0-8ccc297568fd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:40.752893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 543/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:54:40 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 16:54:42.929964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db6222-a6bd-4421-a9ea-a791d362fec3 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:42.930492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db6222-a6bd-4421-a9ea-a791d362fec3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:42.930492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db6222-a6bd-4421-a9ea-a791d362fec3 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:42.932056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 568/1111] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:42.937854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac0d8f-d680-42cf-8118-8f0216485619 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:42.938301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac0d8f-d680-42cf-8118-8f0216485619 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:42.938406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac0d8f-d680-42cf-8118-8f0216485619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:42.960666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a2ac0d8f-d680-42cf-8118-8f0216485619 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:42.962334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a2ac0d8f-d680-42cf-8118-8f0216485619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:43.027602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 544/1112] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:43.304955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db6222-a6bd-4421-a9ea-a791d362fec3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:43.389313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:43.405770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db6222-a6bd-4421-a9ea-a791d362fec3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:43.440253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2205bc5-f812-40af-bb15-17321784d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:43.440868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2205bc5-f812-40af-bb15-17321784d476 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:43.441144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2205bc5-f812-40af-bb15-17321784d476 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:43.441409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2205bc5-f812-40af-bb15-17321784d476 None admin] PATH_INFO: `/v3/roles/372b98bd7cdc4f7fbd4b812a50380cef` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:43.442769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2205bc5-f812-40af-bb15-17321784d476 None admin] RBAC: Authorizing `identity:get_role(role_id=372b98bd7cdc4f7fbd4b812a50380cef)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:43.444135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2205bc5-f812-40af-bb15-17321784d476 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:43.445437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 569/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:54:43 2022] GET /identity/v3/roles/372b98bd7cdc4f7fbd4b812a50380cef => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:43.452159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac0d8f-d680-42cf-8118-8f0216485619 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:43.463867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:43.552042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac0d8f-d680-42cf-8118-8f0216485619 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:43.596939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8732798-1f5e-4e9d-b846-c89d36f497c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:43.597484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8732798-1f5e-4e9d-b846-c89d36f497c8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:43.597870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8732798-1f5e-4e9d-b846-c89d36f497c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:43.598151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8732798-1f5e-4e9d-b846-c89d36f497c8 None admin] PATH_INFO: `/v3/users/4d8828621fb54d54ab26e222f4011998` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:43.612889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8732798-1f5e-4e9d-b846-c89d36f497c8 None admin] RBAC: Authorizing `identity:get_user(user_id=4d8828621fb54d54ab26e222f4011998)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:43.614329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8732798-1f5e-4e9d-b846-c89d36f497c8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:43.615995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 545/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:54:43 2022] GET /identity/v3/users/4d8828621fb54d54ab26e222f4011998 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:43.622068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2205bc5-f812-40af-bb15-17321784d476 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:43.727677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2205bc5-f812-40af-bb15-17321784d476 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:43.761408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb8089-42b1-4ee4-bce8-cc6efe85dec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:43.762010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb8089-42b1-4ee4-bce8-cc6efe85dec6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:43.762291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb8089-42b1-4ee4-bce8-cc6efe85dec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:43.762565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb8089-42b1-4ee4-bce8-cc6efe85dec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:43.784002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:43.830511 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb8089-42b1-4ee4-bce8-cc6efe85dec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:43.831962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb8089-42b1-4ee4-bce8-cc6efe85dec6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:43.930922 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 570/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:54:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:54:43.936917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8732798-1f5e-4e9d-b846-c89d36f497c8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:44.001968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:44.035737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8732798-1f5e-4e9d-b846-c89d36f497c8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:44.068396 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a3678-6738-428b-98ae-bb569b6379fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:44.069019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a3678-6738-428b-98ae-bb569b6379fd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:44.069300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a3678-6738-428b-98ae-bb569b6379fd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:44.069577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a3678-6738-428b-98ae-bb569b6379fd None admin] PATH_INFO: `/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:44.070681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a3678-6738-428b-98ae-bb569b6379fd None admin] RBAC: Authorizing `identity:get_project(project_id=52118eaf3efc4521a61d5f75e3fa7a3a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:44.072659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a3678-6738-428b-98ae-bb569b6379fd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:44.074265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 546/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:54:43 2022] GET /identity/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:44.080511 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb8089-42b1-4ee4-bce8-cc6efe85dec6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:44.131033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:44.176775 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb8089-42b1-4ee4-bce8-cc6efe85dec6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:44.209543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122c7a1-b870-498a-9dfd-ef635185406d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:44.210055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122c7a1-b870-498a-9dfd-ef635185406d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:44.210329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122c7a1-b870-498a-9dfd-ef635185406d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:44.210599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122c7a1-b870-498a-9dfd-ef635185406d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:44.211952 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122c7a1-b870-498a-9dfd-ef635185406d None admin] RBAC: Adding query filter params (role.id=372b98bd7cdc4f7fbd4b812a50380cef, scope.project.id=52118eaf3efc4521a61d5f75e3fa7a3a, user.id=4d8828621fb54d54ab26e222f4011998) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:44.212255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122c7a1-b870-498a-9dfd-ef635185406d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:44.213654 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122c7a1-b870-498a-9dfd-ef635185406d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:44.218892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 571/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:54:44 2022] GET /identity/v3/role_assignments?user.id=4d8828621fb54d54ab26e222f4011998&scope.project.id=52118eaf3efc4521a61d5f75e3fa7a3a&role.id=372b98bd7cdc4f7fbd4b812a50380cef => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:46.338916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7cd74-f73f-4c3a-b00f-d73a37f5b237 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:46.339479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7cd74-f73f-4c3a-b00f-d73a37f5b237 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:46.339479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7cd74-f73f-4c3a-b00f-d73a37f5b237 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:46.340803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 547/1118] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:46.348213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12611e-9be6-465e-9322-1dfbafc591d8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:46.348636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12611e-9be6-465e-9322-1dfbafc591d8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:46.348748 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12611e-9be6-465e-9322-1dfbafc591d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:46.366829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0c12611e-9be6-465e-9322-1dfbafc591d8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:46.368119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0c12611e-9be6-465e-9322-1dfbafc591d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:46.438506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 572/1119] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:46.709505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7cd74-f73f-4c3a-b00f-d73a37f5b237 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:46.809893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7cd74-f73f-4c3a-b00f-d73a37f5b237 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:46.845841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421098c4-5691-459d-b275-4f72a60ea194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:46.846462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421098c4-5691-459d-b275-4f72a60ea194 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:46.846793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421098c4-5691-459d-b275-4f72a60ea194 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:46.847030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421098c4-5691-459d-b275-4f72a60ea194 None admin] PATH_INFO: `/v3/roles/372b98bd7cdc4f7fbd4b812a50380cef` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:46.848479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421098c4-5691-459d-b275-4f72a60ea194 None admin] RBAC: Authorizing `identity:get_role(role_id=372b98bd7cdc4f7fbd4b812a50380cef)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:46.849979 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421098c4-5691-459d-b275-4f72a60ea194 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:46.851384 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 548/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:54:46 2022] GET /identity/v3/roles/372b98bd7cdc4f7fbd4b812a50380cef => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:46.857408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12611e-9be6-465e-9322-1dfbafc591d8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:46.902873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:46.955052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12611e-9be6-465e-9322-1dfbafc591d8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:47.003045 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4e99f-e826-4115-b38b-4e1363c43b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:47.003703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4e99f-e826-4115-b38b-4e1363c43b5a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:47.003992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4e99f-e826-4115-b38b-4e1363c43b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:47.004272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4e99f-e826-4115-b38b-4e1363c43b5a None admin] PATH_INFO: `/v3/users/4d8828621fb54d54ab26e222f4011998` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:47.005441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4e99f-e826-4115-b38b-4e1363c43b5a None admin] RBAC: Authorizing `identity:get_user(user_id=4d8828621fb54d54ab26e222f4011998)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:47.006886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4e99f-e826-4115-b38b-4e1363c43b5a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:47.008533 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 573/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:54:46 2022] GET /identity/v3/users/4d8828621fb54d54ab26e222f4011998 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:47.014531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421098c4-5691-459d-b275-4f72a60ea194 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:47.018352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:47.110797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421098c4-5691-459d-b275-4f72a60ea194 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:47.143889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927c0df-6496-480b-a816-27387a63c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:47.144502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c0df-6496-480b-a816-27387a63c68f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:47.144778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c0df-6496-480b-a816-27387a63c68f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:47.145046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c0df-6496-480b-a816-27387a63c68f None admin] PATH_INFO: `/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:47.146136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927c0df-6496-480b-a816-27387a63c68f None admin] RBAC: Authorizing `identity:get_project(project_id=52118eaf3efc4521a61d5f75e3fa7a3a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:47.147578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927c0df-6496-480b-a816-27387a63c68f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:47.149201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 549/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:54:47 2022] GET /identity/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:47.155419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4e99f-e826-4115-b38b-4e1363c43b5a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:47.254038 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4e99f-e826-4115-b38b-4e1363c43b5a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:47.267739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:47.287815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa0963-30b7-42f7-8367-7a646df4498a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:47.288464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa0963-30b7-42f7-8367-7a646df4498a None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:47.288741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa0963-30b7-42f7-8367-7a646df4498a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:47.289012 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa0963-30b7-42f7-8367-7a646df4498a None admin] PATH_INFO: `/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a/users/4d8828621fb54d54ab26e222f4011998/roles/372b98bd7cdc4f7fbd4b812a50380cef` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:47.291861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa0963-30b7-42f7-8367-7a646df4498a None admin] RBAC: Authorizing `identity:create_grant(project_id=52118eaf3efc4521a61d5f75e3fa7a3a, user_id=4d8828621fb54d54ab26e222f4011998, role_id=372b98bd7cdc4f7fbd4b812a50380cef)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:47.293286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa0963-30b7-42f7-8367-7a646df4498a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:47.305500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 574/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:54:47 2022] PUT /identity/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a/users/4d8828621fb54d54ab26e222f4011998/roles/372b98bd7cdc4f7fbd4b812a50380cef => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:54:49.441686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93daebd-295e-44b1-8c68-3657fb1c7bc7 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:49.442254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93daebd-295e-44b1-8c68-3657fb1c7bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:49.442254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93daebd-295e-44b1-8c68-3657fb1c7bc7 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:49.443565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 550/1124] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:49.448851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81526099-58b6-4c97-a6ed-9588b992b6c9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:49.449289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81526099-58b6-4c97-a6ed-9588b992b6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:49.449403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81526099-58b6-4c97-a6ed-9588b992b6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:49.467353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-81526099-58b6-4c97-a6ed-9588b992b6c9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:49.468555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-81526099-58b6-4c97-a6ed-9588b992b6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:49.538311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 575/1125] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:49.808906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93daebd-295e-44b1-8c68-3657fb1c7bc7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:49.884790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:49.915390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93daebd-295e-44b1-8c68-3657fb1c7bc7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:49.949739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139efb3-6475-49b8-9976-a22c03dce538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:49.950366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139efb3-6475-49b8-9976-a22c03dce538 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:49.950642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139efb3-6475-49b8-9976-a22c03dce538 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:49.950914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139efb3-6475-49b8-9976-a22c03dce538 None admin] PATH_INFO: `/v3/roles/372b98bd7cdc4f7fbd4b812a50380cef` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:49.952320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139efb3-6475-49b8-9976-a22c03dce538 None admin] RBAC: Authorizing `identity:get_role(role_id=372b98bd7cdc4f7fbd4b812a50380cef)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:49.953674 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139efb3-6475-49b8-9976-a22c03dce538 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:49.955003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 551/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:54:49 2022] GET /identity/v3/roles/372b98bd7cdc4f7fbd4b812a50380cef => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:49.960963 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81526099-58b6-4c97-a6ed-9588b992b6c9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:50.057055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81526099-58b6-4c97-a6ed-9588b992b6c9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:50.059245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:50.091064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1cf39-40bc-45d6-b1a0-f0db408ed4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:50.091687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1cf39-40bc-45d6-b1a0-f0db408ed4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:50.091964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1cf39-40bc-45d6-b1a0-f0db408ed4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:50.092232 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1cf39-40bc-45d6-b1a0-f0db408ed4a6 None admin] PATH_INFO: `/v3/users/4d8828621fb54d54ab26e222f4011998` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:50.093332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1cf39-40bc-45d6-b1a0-f0db408ed4a6 None admin] RBAC: Authorizing `identity:get_user(user_id=4d8828621fb54d54ab26e222f4011998)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:50.094731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1cf39-40bc-45d6-b1a0-f0db408ed4a6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:50.096346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 576/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:54:49 2022] GET /identity/v3/users/4d8828621fb54d54ab26e222f4011998 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:50.102614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139efb3-6475-49b8-9976-a22c03dce538 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:50.207181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139efb3-6475-49b8-9976-a22c03dce538 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:50.241540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa38da6-ab5c-4ee9-9a33-14c709b5558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:50.242145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa38da6-ab5c-4ee9-9a33-14c709b5558c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:50.242981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa38da6-ab5c-4ee9-9a33-14c709b5558c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:50.243250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa38da6-ab5c-4ee9-9a33-14c709b5558c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:50.253912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:50.311196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa38da6-ab5c-4ee9-9a33-14c709b5558c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:50.312562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa38da6-ab5c-4ee9-9a33-14c709b5558c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:50.411917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 552/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:54:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:54:50.418148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1cf39-40bc-45d6-b1a0-f0db408ed4a6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:50.515999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1cf39-40bc-45d6-b1a0-f0db408ed4a6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:50.550018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fe4e2-c65b-488d-a67d-97157dc83f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:50.550638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fe4e2-c65b-488d-a67d-97157dc83f48 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:50.550914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fe4e2-c65b-488d-a67d-97157dc83f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:50.551215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fe4e2-c65b-488d-a67d-97157dc83f48 None admin] PATH_INFO: `/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:50.552318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fe4e2-c65b-488d-a67d-97157dc83f48 None admin] RBAC: Authorizing `identity:get_project(project_id=52118eaf3efc4521a61d5f75e3fa7a3a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:50.553727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fe4e2-c65b-488d-a67d-97157dc83f48 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:50.555909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 577/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:54:50 2022] GET /identity/v3/projects/52118eaf3efc4521a61d5f75e3fa7a3a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:54:50.562350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa38da6-ab5c-4ee9-9a33-14c709b5558c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:50.603517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:50.661545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa38da6-ab5c-4ee9-9a33-14c709b5558c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:50.695976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45dd662-8613-4d27-ba96-3a6f17db42bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:50.696573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45dd662-8613-4d27-ba96-3a6f17db42bb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:50.696895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45dd662-8613-4d27-ba96-3a6f17db42bb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:50.697115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45dd662-8613-4d27-ba96-3a6f17db42bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:50.697827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45dd662-8613-4d27-ba96-3a6f17db42bb None admin] RBAC: Adding query filter params (role.id=372b98bd7cdc4f7fbd4b812a50380cef, scope.project.id=52118eaf3efc4521a61d5f75e3fa7a3a, user.id=4d8828621fb54d54ab26e222f4011998) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:50.698130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45dd662-8613-4d27-ba96-3a6f17db42bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:50.699586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45dd662-8613-4d27-ba96-3a6f17db42bb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:50.705535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 553/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:54:50 2022] GET /identity/v3/role_assignments?user.id=4d8828621fb54d54ab26e222f4011998&scope.project.id=52118eaf3efc4521a61d5f75e3fa7a3a&role.id=372b98bd7cdc4f7fbd4b812a50380cef => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:52.893263 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904d5d-fcdb-4c93-81d8-77b1d4b22c93 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:52.893772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904d5d-fcdb-4c93-81d8-77b1d4b22c93 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:52.893900 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904d5d-fcdb-4c93-81d8-77b1d4b22c93 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:52.895185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 578/1131] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:52.900412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdf8d6-d011-4cbf-80be-e4ebd94246dc None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:52.900846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdf8d6-d011-4cbf-80be-e4ebd94246dc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:52.900947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdf8d6-d011-4cbf-80be-e4ebd94246dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:52.919148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-cbfdf8d6-d011-4cbf-80be-e4ebd94246dc None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:52.920319 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cbfdf8d6-d011-4cbf-80be-e4ebd94246dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:52.985225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 554/1132] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:53.252216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc904d5d-fcdb-4c93-81d8-77b1d4b22c93 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:53.295271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:53.352196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc904d5d-fcdb-4c93-81d8-77b1d4b22c93 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:53.386246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff571e-bb62-4f94-9805-4df6b72bb6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:53.386868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff571e-bb62-4f94-9805-4df6b72bb6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:53.387299 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff571e-bb62-4f94-9805-4df6b72bb6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:53.387577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff571e-bb62-4f94-9805-4df6b72bb6a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:53.388694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff571e-bb62-4f94-9805-4df6b72bb6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:53.390113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff571e-bb62-4f94-9805-4df6b72bb6a5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:53.391711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 579/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:53.397504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdf8d6-d011-4cbf-80be-e4ebd94246dc None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:53.430598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:53.496172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdf8d6-d011-4cbf-80be-e4ebd94246dc None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:53.534378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14292d7d-68f5-4c4f-9240-598efb52dc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:53.534897 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292d7d-68f5-4c4f-9240-598efb52dc9e None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:53.535197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292d7d-68f5-4c4f-9240-598efb52dc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:53.535462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292d7d-68f5-4c4f-9240-598efb52dc9e None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:53.536112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14292d7d-68f5-4c4f-9240-598efb52dc9e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:53.537534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14292d7d-68f5-4c4f-9240-598efb52dc9e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:53.564561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 555/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:54:53 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:55.754190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ae0ec-ff79-4bc4-be7a-4d7b5fbd9299 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:55.754766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ae0ec-ff79-4bc4-be7a-4d7b5fbd9299 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:55.754766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ae0ec-ff79-4bc4-be7a-4d7b5fbd9299 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:55.756100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 580/1135] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:55.761653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522d5f9-10a9-416d-bdab-206a75901d13 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:55.762359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522d5f9-10a9-416d-bdab-206a75901d13 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:55.762359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522d5f9-10a9-416d-bdab-206a75901d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:55.781152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1522d5f9-10a9-416d-bdab-206a75901d13 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:55.782342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1522d5f9-10a9-416d-bdab-206a75901d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:55.848581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 556/1136] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:56.133189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ae0ec-ff79-4bc4-be7a-4d7b5fbd9299 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:56.233128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ae0ec-ff79-4bc4-be7a-4d7b5fbd9299 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:56.243253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:56.268046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cdc00-fda6-4551-a08d-3a1bb7b00d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:56.268666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cdc00-fda6-4551-a08d-3a1bb7b00d37 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:56.268940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cdc00-fda6-4551-a08d-3a1bb7b00d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:56.269206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cdc00-fda6-4551-a08d-3a1bb7b00d37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:56.270291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cdc00-fda6-4551-a08d-3a1bb7b00d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:56.271719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cdc00-fda6-4551-a08d-3a1bb7b00d37 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:56.273288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 581/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:54:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:56.279567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522d5f9-10a9-416d-bdab-206a75901d13 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:56.297182 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:56.380054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522d5f9-10a9-416d-bdab-206a75901d13 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:56.413322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f2609-5625-4263-a596-091731d2b8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:56.413860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f2609-5625-4263-a596-091731d2b8b6 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:56.414757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f2609-5625-4263-a596-091731d2b8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:56.415060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f2609-5625-4263-a596-091731d2b8b6 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:56.415726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f2609-5625-4263-a596-091731d2b8b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:56.417155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f2609-5625-4263-a596-091731d2b8b6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:56.437718 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 557/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:54:56 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:54:58.635587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90196ecc-a7d5-42c6-b2c9-a416585162e2 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:58.636122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90196ecc-a7d5-42c6-b2c9-a416585162e2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:58.636122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90196ecc-a7d5-42c6-b2c9-a416585162e2 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:58.637467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 582/1139] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:54:58.642783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2d90d-30ac-43b1-8b83-7639a3308a29 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:58.643241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2d90d-30ac-43b1-8b83-7639a3308a29 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:58.643358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2d90d-30ac-43b1-8b83-7639a3308a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:58.661675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-79b2d90d-30ac-43b1-8b83-7639a3308a29 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:54:58.662838 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-79b2d90d-30ac-43b1-8b83-7639a3308a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:54:58.740223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 558/1140] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:54:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:54:59.008658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90196ecc-a7d5-42c6-b2c9-a416585162e2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:59.113141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90196ecc-a7d5-42c6-b2c9-a416585162e2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:59.145935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a54ec-6970-4501-9cb5-74ae5dcd7b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:59.146557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a54ec-6970-4501-9cb5-74ae5dcd7b3b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:59.146835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a54ec-6970-4501-9cb5-74ae5dcd7b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:59.147135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a54ec-6970-4501-9cb5-74ae5dcd7b3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:59.157409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a54ec-6970-4501-9cb5-74ae5dcd7b3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-df7a54ec-6970-4501-9cb5-74ae5dcd7b3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:54:59.162178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:54:59.163401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:54:59.164477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 583/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:54:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:54:59.169400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2d90d-30ac-43b1-8b83-7639a3308a29 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:59.171583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:59.268496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2d90d-30ac-43b1-8b83-7639a3308a29 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:59.302314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea44f0-0153-4630-aa70-e1be402d7215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:59.302919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea44f0-0153-4630-aa70-e1be402d7215 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:59.303234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea44f0-0153-4630-aa70-e1be402d7215 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:59.303509 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea44f0-0153-4630-aa70-e1be402d7215 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:59.304164 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea44f0-0153-4630-aa70-e1be402d7215 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:54:59.304468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea44f0-0153-4630-aa70-e1be402d7215 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:59.305832 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea44f0-0153-4630-aa70-e1be402d7215 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:59.313611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 559/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:54:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:59.319595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a54ec-6970-4501-9cb5-74ae5dcd7b3b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:59.360082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:59.424620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a54ec-6970-4501-9cb5-74ae5dcd7b3b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:59.458004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b85d84-06e1-4b3e-9833-62eeaf262490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:59.458620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b85d84-06e1-4b3e-9833-62eeaf262490 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:59.458901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b85d84-06e1-4b3e-9833-62eeaf262490 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:59.459203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b85d84-06e1-4b3e-9833-62eeaf262490 None admin] PATH_INFO: `/v3/users/37b7087b9e9b4b3f8a2038051d304928` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:59.471816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b85d84-06e1-4b3e-9833-62eeaf262490 None admin] RBAC: Authorizing `identity:get_user(user_id=37b7087b9e9b4b3f8a2038051d304928)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:59.473238 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b85d84-06e1-4b3e-9833-62eeaf262490 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:59.474827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 584/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:54:59 2022] GET /identity/v3/users/37b7087b9e9b4b3f8a2038051d304928 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:59.480888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea44f0-0153-4630-aa70-e1be402d7215 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:59.541847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:59.583838 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea44f0-0153-4630-aa70-e1be402d7215 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:59.619662 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ad77c-e60e-42b2-8092-41f7ab53abf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:59.620259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad77c-e60e-42b2-8092-41f7ab53abf5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:59.620542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad77c-e60e-42b2-8092-41f7ab53abf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:59.620816 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad77c-e60e-42b2-8092-41f7ab53abf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:59.687034 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ad77c-e60e-42b2-8092-41f7ab53abf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:59.688478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ad77c-e60e-42b2-8092-41f7ab53abf5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:59.746447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:59.784247 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 560/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:54:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:54:59.790796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b85d84-06e1-4b3e-9833-62eeaf262490 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:54:59.880599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:54:59.888256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b85d84-06e1-4b3e-9833-62eeaf262490 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:54:59.921153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e02d61-7277-4914-8809-150da9f8ad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:54:59.921779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e02d61-7277-4914-8809-150da9f8ad13 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:54:59.922051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e02d61-7277-4914-8809-150da9f8ad13 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:54:59.922318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e02d61-7277-4914-8809-150da9f8ad13 None admin] PATH_INFO: `/v3/projects/66280f8839cc4a6db5e189f2daa54766` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:54:59.923705 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e02d61-7277-4914-8809-150da9f8ad13 None admin] RBAC: Authorizing `identity:get_project(project_id=66280f8839cc4a6db5e189f2daa54766)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:54:59.925111 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e02d61-7277-4914-8809-150da9f8ad13 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:54:59.926720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 585/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:54:59 2022] GET /identity/v3/projects/66280f8839cc4a6db5e189f2daa54766 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:54:59.932910 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ad77c-e60e-42b2-8092-41f7ab53abf5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:00.028881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ad77c-e60e-42b2-8092-41f7ab53abf5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:00.061638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ec00b-9025-4ee0-a266-dfa22bfaf841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:00.062229 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ec00b-9025-4ee0-a266-dfa22bfaf841 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:00.062503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ec00b-9025-4ee0-a266-dfa22bfaf841 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:00.062771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ec00b-9025-4ee0-a266-dfa22bfaf841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:00.063517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ec00b-9025-4ee0-a266-dfa22bfaf841 None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=66280f8839cc4a6db5e189f2daa54766, user.id=37b7087b9e9b4b3f8a2038051d304928) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:55:00.063823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ec00b-9025-4ee0-a266-dfa22bfaf841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:00.065243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ec00b-9025-4ee0-a266-dfa22bfaf841 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:00.071056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 561/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:54:59 2022] GET /identity/v3/role_assignments?user.id=37b7087b9e9b4b3f8a2038051d304928&scope.project.id=66280f8839cc4a6db5e189f2daa54766&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:02.171608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649350-da8b-4e67-aa22-f5e9065324a3 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:02.172161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649350-da8b-4e67-aa22-f5e9065324a3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:02.172161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649350-da8b-4e67-aa22-f5e9065324a3 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:02.173662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 586/1147] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:55:02.178904 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79336dca-ac5b-44eb-8ed7-1478cb951fbb None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:02.179449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79336dca-ac5b-44eb-8ed7-1478cb951fbb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:02.179553 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79336dca-ac5b-44eb-8ed7-1478cb951fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:02.197905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-79336dca-ac5b-44eb-8ed7-1478cb951fbb None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:55:02.199122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-79336dca-ac5b-44eb-8ed7-1478cb951fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:55:02.221659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:02.267121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 562/1148] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:55:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:55:02.540196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d649350-da8b-4e67-aa22-f5e9065324a3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:02.646572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d649350-da8b-4e67-aa22-f5e9065324a3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:02.682477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ff1c1-8bc9-4a38-a544-df6357cfe090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:02.683136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ff1c1-8bc9-4a38-a544-df6357cfe090 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:02.683411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ff1c1-8bc9-4a38-a544-df6357cfe090 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:02.683681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ff1c1-8bc9-4a38-a544-df6357cfe090 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:02.693717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ff1c1-8bc9-4a38-a544-df6357cfe090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-817ff1c1-8bc9-4a38-a544-df6357cfe090 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:55:02.698481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:55:02.699715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:55:02.700827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 587/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:55:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:55:02.705661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79336dca-ac5b-44eb-8ed7-1478cb951fbb None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:02.812270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79336dca-ac5b-44eb-8ed7-1478cb951fbb None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:02.845180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e52ebb-b2fb-470c-aeb6-ee43b83bd45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:02.845785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e52ebb-b2fb-470c-aeb6-ee43b83bd45d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:02.846067 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e52ebb-b2fb-470c-aeb6-ee43b83bd45d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:02.846340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e52ebb-b2fb-470c-aeb6-ee43b83bd45d None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:02.847029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e52ebb-b2fb-470c-aeb6-ee43b83bd45d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:55:02.847337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e52ebb-b2fb-470c-aeb6-ee43b83bd45d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:02.848714 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e52ebb-b2fb-470c-aeb6-ee43b83bd45d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:02.857755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 563/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:55:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:02.863624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ff1c1-8bc9-4a38-a544-df6357cfe090 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:02.888607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:02.964447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ff1c1-8bc9-4a38-a544-df6357cfe090 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:02.999041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bccab24-07ed-41d0-8f87-c046358c6aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:02.999667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccab24-07ed-41d0-8f87-c046358c6aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:02.999944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccab24-07ed-41d0-8f87-c046358c6aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:03.000214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccab24-07ed-41d0-8f87-c046358c6aa2 None admin] PATH_INFO: `/v3/users/37b7087b9e9b4b3f8a2038051d304928` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:03.001350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccab24-07ed-41d0-8f87-c046358c6aa2 None admin] RBAC: Authorizing `identity:get_user(user_id=37b7087b9e9b4b3f8a2038051d304928)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:03.002774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccab24-07ed-41d0-8f87-c046358c6aa2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:03.004391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 588/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:55:02 2022] GET /identity/v3/users/37b7087b9e9b4b3f8a2038051d304928 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:03.011130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e52ebb-b2fb-470c-aeb6-ee43b83bd45d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:03.033099 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:03.111392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e52ebb-b2fb-470c-aeb6-ee43b83bd45d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:03.144605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3c79c-4e86-41e7-a2ae-b684bf831650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:03.145231 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3c79c-4e86-41e7-a2ae-b684bf831650 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:03.145520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3c79c-4e86-41e7-a2ae-b684bf831650 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:03.145773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3c79c-4e86-41e7-a2ae-b684bf831650 None admin] PATH_INFO: `/v3/projects/66280f8839cc4a6db5e189f2daa54766` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:03.146874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3c79c-4e86-41e7-a2ae-b684bf831650 None admin] RBAC: Authorizing `identity:get_project(project_id=66280f8839cc4a6db5e189f2daa54766)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:03.148897 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3c79c-4e86-41e7-a2ae-b684bf831650 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:03.150507 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 564/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:55:03 2022] GET /identity/v3/projects/66280f8839cc4a6db5e189f2daa54766 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:03.156723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bccab24-07ed-41d0-8f87-c046358c6aa2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:03.248448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:03.258975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bccab24-07ed-41d0-8f87-c046358c6aa2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:03.293643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a86669-8d4d-4bff-b83a-722e227aa64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:03.294292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a86669-8d4d-4bff-b83a-722e227aa64a None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:03.294574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a86669-8d4d-4bff-b83a-722e227aa64a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:03.294848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a86669-8d4d-4bff-b83a-722e227aa64a None admin] PATH_INFO: `/v3/projects/66280f8839cc4a6db5e189f2daa54766/users/37b7087b9e9b4b3f8a2038051d304928/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:03.296790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a86669-8d4d-4bff-b83a-722e227aa64a None admin] RBAC: Authorizing `identity:create_grant(project_id=66280f8839cc4a6db5e189f2daa54766, user_id=37b7087b9e9b4b3f8a2038051d304928, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:03.298203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a86669-8d4d-4bff-b83a-722e227aa64a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:03.311505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 589/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:55:03 2022] PUT /identity/v3/projects/66280f8839cc4a6db5e189f2daa54766/users/37b7087b9e9b4b3f8a2038051d304928/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:55:05.457363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d3e2f-3f3b-4c15-943f-e70131a852dd None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:05.457923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d3e2f-3f3b-4c15-943f-e70131a852dd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:05.457923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d3e2f-3f3b-4c15-943f-e70131a852dd None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:05.459273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 565/1154] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:55:05.464703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cce832-49a2-46ff-8957-2a6d55432ac2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:05.465073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cce832-49a2-46ff-8957-2a6d55432ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:05.465254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cce832-49a2-46ff-8957-2a6d55432ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:05.482991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-45cce832-49a2-46ff-8957-2a6d55432ac2 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:55:05.484178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-45cce832-49a2-46ff-8957-2a6d55432ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:55:05.548892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 590/1155] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:55:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:55:05.833472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d3e2f-3f3b-4c15-943f-e70131a852dd None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:05.926283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:05.939634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d3e2f-3f3b-4c15-943f-e70131a852dd None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:05.973116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf57f0-2fee-4ead-a52e-5ec1d903afc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:05.973737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf57f0-2fee-4ead-a52e-5ec1d903afc5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:05.974017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf57f0-2fee-4ead-a52e-5ec1d903afc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:05.974290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf57f0-2fee-4ead-a52e-5ec1d903afc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:05.984782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf57f0-2fee-4ead-a52e-5ec1d903afc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-80bf57f0-2fee-4ead-a52e-5ec1d903afc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:55:05.989579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:55:05.990658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:55:05.992140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 566/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:55:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:55:05.997293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cce832-49a2-46ff-8957-2a6d55432ac2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:06.085990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:06.109889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cce832-49a2-46ff-8957-2a6d55432ac2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:06.145318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef84baf-3c68-47d6-be82-4acbe11f2c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:06.145983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef84baf-3c68-47d6-be82-4acbe11f2c55 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:06.146267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef84baf-3c68-47d6-be82-4acbe11f2c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:06.146547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef84baf-3c68-47d6-be82-4acbe11f2c55 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:06.147239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef84baf-3c68-47d6-be82-4acbe11f2c55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:55:06.147541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef84baf-3c68-47d6-be82-4acbe11f2c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:06.148932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef84baf-3c68-47d6-be82-4acbe11f2c55 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:06.157081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 591/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:55:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:06.163220 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf57f0-2fee-4ead-a52e-5ec1d903afc5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:06.269748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf57f0-2fee-4ead-a52e-5ec1d903afc5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:06.303330 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7be98-8246-4fe8-b113-ddeafedc9285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:06.303918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7be98-8246-4fe8-b113-ddeafedc9285 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:06.304742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7be98-8246-4fe8-b113-ddeafedc9285 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:06.305021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7be98-8246-4fe8-b113-ddeafedc9285 None admin] PATH_INFO: `/v3/users/37b7087b9e9b4b3f8a2038051d304928` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:06.306131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7be98-8246-4fe8-b113-ddeafedc9285 None admin] RBAC: Authorizing `identity:get_user(user_id=37b7087b9e9b4b3f8a2038051d304928)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:06.307603 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7be98-8246-4fe8-b113-ddeafedc9285 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:06.309170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 567/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:55:06 2022] GET /identity/v3/users/37b7087b9e9b4b3f8a2038051d304928 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:06.315053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef84baf-3c68-47d6-be82-4acbe11f2c55 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:06.413491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef84baf-3c68-47d6-be82-4acbe11f2c55 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:06.446177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487db13-d6d9-481b-a306-0a6df680d885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:06.446766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487db13-d6d9-481b-a306-0a6df680d885 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:06.447075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487db13-d6d9-481b-a306-0a6df680d885 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:06.447345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487db13-d6d9-481b-a306-0a6df680d885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:06.458508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:06.512552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487db13-d6d9-481b-a306-0a6df680d885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:06.513988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487db13-d6d9-481b-a306-0a6df680d885 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:06.608878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 592/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:55:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:55:06.614841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7be98-8246-4fe8-b113-ddeafedc9285 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:06.634791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:06.715751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7be98-8246-4fe8-b113-ddeafedc9285 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:06.748654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c0968-ced0-47e6-95ee-58b0bb46a689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:06.749260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c0968-ced0-47e6-95ee-58b0bb46a689 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:06.749539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c0968-ced0-47e6-95ee-58b0bb46a689 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:06.749811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c0968-ced0-47e6-95ee-58b0bb46a689 None admin] PATH_INFO: `/v3/projects/66280f8839cc4a6db5e189f2daa54766` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:06.750930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c0968-ced0-47e6-95ee-58b0bb46a689 None admin] RBAC: Authorizing `identity:get_project(project_id=66280f8839cc4a6db5e189f2daa54766)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:06.752393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c0968-ced0-47e6-95ee-58b0bb46a689 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:06.754000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 568/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:55:06 2022] GET /identity/v3/projects/66280f8839cc4a6db5e189f2daa54766 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:06.760091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487db13-d6d9-481b-a306-0a6df680d885 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:06.803485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:06.864348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487db13-d6d9-481b-a306-0a6df680d885 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:06.897203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081dde9-1333-408f-84bf-3cbcc6cbc98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:06.897792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081dde9-1333-408f-84bf-3cbcc6cbc98e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:06.898585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081dde9-1333-408f-84bf-3cbcc6cbc98e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:06.898859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081dde9-1333-408f-84bf-3cbcc6cbc98e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:06.899598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081dde9-1333-408f-84bf-3cbcc6cbc98e None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=66280f8839cc4a6db5e189f2daa54766, user.id=37b7087b9e9b4b3f8a2038051d304928) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:55:06.899901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081dde9-1333-408f-84bf-3cbcc6cbc98e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:06.901319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081dde9-1333-408f-84bf-3cbcc6cbc98e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:06.907206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 593/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:55:06 2022] GET /identity/v3/role_assignments?user.id=37b7087b9e9b4b3f8a2038051d304928&scope.project.id=66280f8839cc4a6db5e189f2daa54766&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:09.075817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9f365-9654-487f-a2bc-0b7893f5f72a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:09.076342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9f365-9654-487f-a2bc-0b7893f5f72a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:09.076342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9f365-9654-487f-a2bc-0b7893f5f72a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:09.077804 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 569/1162] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:55:09.084045 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44354db8-d2dd-4ac7-a5b7-9714c2359b61 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:09.084628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44354db8-d2dd-4ac7-a5b7-9714c2359b61 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:09.084628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44354db8-d2dd-4ac7-a5b7-9714c2359b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:09.104146 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-44354db8-d2dd-4ac7-a5b7-9714c2359b61 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:55:09.105369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-44354db8-d2dd-4ac7-a5b7-9714c2359b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:55:09.173928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 594/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:55:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:55:09.439986 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9f365-9654-487f-a2bc-0b7893f5f72a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:09.528338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:09.545531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9f365-9654-487f-a2bc-0b7893f5f72a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:09.579047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36c28f-feb8-4a40-adc2-2b4786d9df6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:09.579650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c28f-feb8-4a40-adc2-2b4786d9df6f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:09.579926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c28f-feb8-4a40-adc2-2b4786d9df6f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:09.580193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c28f-feb8-4a40-adc2-2b4786d9df6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:09.646272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36c28f-feb8-4a40-adc2-2b4786d9df6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:09.647767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36c28f-feb8-4a40-adc2-2b4786d9df6f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:09.732494 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:09.750351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 570/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:55:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:55:09.756502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44354db8-d2dd-4ac7-a5b7-9714c2359b61 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:09.791040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:09.855448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44354db8-d2dd-4ac7-a5b7-9714c2359b61 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:09.888905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ad9f9-9057-47df-8a1c-ef16c2e7df0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:09.889531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ad9f9-9057-47df-8a1c-ef16c2e7df0a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:09.889810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ad9f9-9057-47df-8a1c-ef16c2e7df0a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:09.890083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ad9f9-9057-47df-8a1c-ef16c2e7df0a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:09.896619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ad9f9-9057-47df-8a1c-ef16c2e7df0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:09.898542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ad9f9-9057-47df-8a1c-ef16c2e7df0a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b41ad9f9-9057-47df-8a1c-ef16c2e7df0a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:55:09.909965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:55:09.911203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:55:09.912080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 595/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 22 16:55:09 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:55:09.917903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36c28f-feb8-4a40-adc2-2b4786d9df6f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:10.017752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36c28f-feb8-4a40-adc2-2b4786d9df6f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:10.050989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b480825-648d-4d08-b157-ebd7e4e90952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:10.051605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b480825-648d-4d08-b157-ebd7e4e90952 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:10.051886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b480825-648d-4d08-b157-ebd7e4e90952 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:10.052157 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b480825-648d-4d08-b157-ebd7e4e90952 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:10.052796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b480825-648d-4d08-b157-ebd7e4e90952 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:55:10.053095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b480825-648d-4d08-b157-ebd7e4e90952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:10.055015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b480825-648d-4d08-b157-ebd7e4e90952 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:10.062272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 571/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 22 16:55:09 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:12.213649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46644bfe-28d1-4daa-af09-f41d86d56d32 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:12.214169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46644bfe-28d1-4daa-af09-f41d86d56d32 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:12.214169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46644bfe-28d1-4daa-af09-f41d86d56d32 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:12.215540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 596/1167] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:55:12.220855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d79d-2a13-419c-aa52-7894653066a0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:12.221243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d79d-2a13-419c-aa52-7894653066a0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:12.221409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d79d-2a13-419c-aa52-7894653066a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:12.239812 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-fc30d79d-2a13-419c-aa52-7894653066a0 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:55:12.241024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fc30d79d-2a13-419c-aa52-7894653066a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:55:12.283492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:12.307403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 572/1168] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:55:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:55:12.579140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46644bfe-28d1-4daa-af09-f41d86d56d32 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:12.672560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:12.680054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46644bfe-28d1-4daa-af09-f41d86d56d32 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:12.713560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f92dba-9b30-4c0f-8566-4ec7e98c77af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:12.714161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f92dba-9b30-4c0f-8566-4ec7e98c77af None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:12.714443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f92dba-9b30-4c0f-8566-4ec7e98c77af None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:12.714719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f92dba-9b30-4c0f-8566-4ec7e98c77af None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:12.715421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f92dba-9b30-4c0f-8566-4ec7e98c77af None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:12.716770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f92dba-9b30-4c0f-8566-4ec7e98c77af None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:12.733149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 597/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Thu Sep 22 16:55:12 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:55:14.926640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d902df-b0a5-482a-b22d-b8af631203c5 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:14.927235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d902df-b0a5-482a-b22d-b8af631203c5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:14.927235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d902df-b0a5-482a-b22d-b8af631203c5 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:14.928522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 573/1170] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:55:14.933708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0ad8-14b8-4405-8057-2dafdb174fa9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:14.934090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0ad8-14b8-4405-8057-2dafdb174fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:14.934248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0ad8-14b8-4405-8057-2dafdb174fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:14.952469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f05a0ad8-14b8-4405-8057-2dafdb174fa9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:55:14.953623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f05a0ad8-14b8-4405-8057-2dafdb174fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:55:15.019475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 598/1171] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:55:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:55:15.286341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d902df-b0a5-482a-b22d-b8af631203c5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:15.385607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d902df-b0a5-482a-b22d-b8af631203c5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:15.418410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70c58d-8e18-4277-83db-64cbb9109b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:15.419050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70c58d-8e18-4277-83db-64cbb9109b6a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:15.419337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70c58d-8e18-4277-83db-64cbb9109b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:15.419601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70c58d-8e18-4277-83db-64cbb9109b6a None admin] PATH_INFO: `/v3/domains/1443216a64a64d8f9dd92d218989fef9` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:15.430431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70c58d-8e18-4277-83db-64cbb9109b6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=1443216a64a64d8f9dd92d218989fef9)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:15.431890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70c58d-8e18-4277-83db-64cbb9109b6a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:15.433432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 574/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Sep 22 16:55:15 2022] GET /identity/v3/domains/1443216a64a64d8f9dd92d218989fef9 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:15.439258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a0ad8-14b8-4405-8057-2dafdb174fa9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:15.519258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:15.534343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a0ad8-14b8-4405-8057-2dafdb174fa9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:15.567870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3833222-d30a-4da8-82af-74862d0ca22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:15.568392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3833222-d30a-4da8-82af-74862d0ca22f None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:15.568664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3833222-d30a-4da8-82af-74862d0ca22f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:15.568930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3833222-d30a-4da8-82af-74862d0ca22f None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:15.570060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3833222-d30a-4da8-82af-74862d0ca22f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:15.571486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3833222-d30a-4da8-82af-74862d0ca22f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:15.596596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 599/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:55:15 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:55:17.771629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a91d25-4d81-46da-adca-442ddb6f368d None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:17.772066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a91d25-4d81-46da-adca-442ddb6f368d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:17.772166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a91d25-4d81-46da-adca-442ddb6f368d None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:17.773498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 575/1174] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:55:17.779311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfb242-cd1d-494d-8084-90bece5afdd0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:17.779733 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfb242-cd1d-494d-8084-90bece5afdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:17.779848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfb242-cd1d-494d-8084-90bece5afdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:17.798121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-94dfb242-cd1d-494d-8084-90bece5afdd0 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:55:17.799570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-94dfb242-cd1d-494d-8084-90bece5afdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:55:17.863465 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 600/1175] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:55:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:55:18.134537 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a91d25-4d81-46da-adca-442ddb6f368d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:18.177677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:18.241060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a91d25-4d81-46da-adca-442ddb6f368d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:18.278789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b57c5-e92f-4878-90a6-891d7f69059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:18.279655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b57c5-e92f-4878-90a6-891d7f69059f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:18.279969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b57c5-e92f-4878-90a6-891d7f69059f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:18.280218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b57c5-e92f-4878-90a6-891d7f69059f None admin] PATH_INFO: `/v3/domains/1443216a64a64d8f9dd92d218989fef9` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:18.282049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b57c5-e92f-4878-90a6-891d7f69059f None admin] RBAC: Authorizing `identity:get_domain(domain_id=1443216a64a64d8f9dd92d218989fef9)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:18.284245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b57c5-e92f-4878-90a6-891d7f69059f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:18.286619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 576/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Sep 22 16:55:18 2022] GET /identity/v3/domains/1443216a64a64d8f9dd92d218989fef9 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:18.293381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfb242-cd1d-494d-8084-90bece5afdd0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:18.363451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:18.403142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfb242-cd1d-494d-8084-90bece5afdd0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:18.438450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03968f45-de08-430c-9820-1bb1c2cde26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:18.439077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03968f45-de08-430c-9820-1bb1c2cde26c None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:18.439359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03968f45-de08-430c-9820-1bb1c2cde26c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:18.439631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03968f45-de08-430c-9820-1bb1c2cde26c None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:18.440275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03968f45-de08-430c-9820-1bb1c2cde26c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:18.441693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03968f45-de08-430c-9820-1bb1c2cde26c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:18.465413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 601/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:55:18 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:55:20.657919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671faa23-378a-452a-870a-d43f6c119649 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:20.658520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671faa23-378a-452a-870a-d43f6c119649 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:20.658520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671faa23-378a-452a-870a-d43f6c119649 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:20.659810 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 577/1178] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:55:20.665764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a11b-311c-4b8d-b57a-202d32306470 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:20.666201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a11b-311c-4b8d-b57a-202d32306470 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:20.666372 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a11b-311c-4b8d-b57a-202d32306470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:20.686081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ec17a11b-311c-4b8d-b57a-202d32306470 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:55:20.687317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ec17a11b-311c-4b8d-b57a-202d32306470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:55:20.755775 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 602/1179] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:55:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:55:21.027126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671faa23-378a-452a-870a-d43f6c119649 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:21.125540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671faa23-378a-452a-870a-d43f6c119649 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:21.132647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:21.160960 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281a9cb-f072-4e37-a56f-d8d3f829f7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:21.161590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281a9cb-f072-4e37-a56f-d8d3f829f7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:21.161870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281a9cb-f072-4e37-a56f-d8d3f829f7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:21.162143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281a9cb-f072-4e37-a56f-d8d3f829f7c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:21.172570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281a9cb-f072-4e37-a56f-d8d3f829f7c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-5281a9cb-f072-4e37-a56f-d8d3f829f7c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:55:21.177690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:55:21.178808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:55:21.180020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 578/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:55:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:55:21.184846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17a11b-311c-4b8d-b57a-202d32306470 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:21.238122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:21.287162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17a11b-311c-4b8d-b57a-202d32306470 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:21.320040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32213384-1d8b-4257-b912-a850c8693ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:21.320641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32213384-1d8b-4257-b912-a850c8693ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:21.320921 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32213384-1d8b-4257-b912-a850c8693ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:21.321195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32213384-1d8b-4257-b912-a850c8693ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:21.321851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32213384-1d8b-4257-b912-a850c8693ab8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:55:21.322154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32213384-1d8b-4257-b912-a850c8693ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:21.323544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32213384-1d8b-4257-b912-a850c8693ab8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:21.331429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 603/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:55:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:21.338031 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281a9cb-f072-4e37-a56f-d8d3f829f7c1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:21.444554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281a9cb-f072-4e37-a56f-d8d3f829f7c1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:21.478367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d049e-141b-4216-809a-72c5b51037e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:21.479017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d049e-141b-4216-809a-72c5b51037e9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:21.479300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d049e-141b-4216-809a-72c5b51037e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:21.479574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d049e-141b-4216-809a-72c5b51037e9 None admin] PATH_INFO: `/v3/users/6069ff6f26f443cbad983ad80c04b297` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:21.492207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d049e-141b-4216-809a-72c5b51037e9 None admin] RBAC: Authorizing `identity:get_user(user_id=6069ff6f26f443cbad983ad80c04b297)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:21.493647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d049e-141b-4216-809a-72c5b51037e9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:21.495260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 579/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:55:21 2022] GET /identity/v3/users/6069ff6f26f443cbad983ad80c04b297 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:21.501174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32213384-1d8b-4257-b912-a850c8693ab8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:21.598297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32213384-1d8b-4257-b912-a850c8693ab8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:21.614259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:21.631859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc21fc-5271-4bb8-b3d6-775d825d3bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:21.632458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc21fc-5271-4bb8-b3d6-775d825d3bca None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:21.632738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc21fc-5271-4bb8-b3d6-775d825d3bca None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:21.633013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc21fc-5271-4bb8-b3d6-775d825d3bca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:21.698823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc21fc-5271-4bb8-b3d6-775d825d3bca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:21.700281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc21fc-5271-4bb8-b3d6-775d825d3bca None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:21.806614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 604/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:55:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:55:21.812794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d049e-141b-4216-809a-72c5b51037e9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:21.836985 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:21.910124 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d049e-141b-4216-809a-72c5b51037e9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:21.944161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b105d18-d5bb-4fd9-9d2f-f5ecb661c908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:21.944783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b105d18-d5bb-4fd9-9d2f-f5ecb661c908 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:21.945063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b105d18-d5bb-4fd9-9d2f-f5ecb661c908 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:21.945339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b105d18-d5bb-4fd9-9d2f-f5ecb661c908 None admin] PATH_INFO: `/v3/projects/f0a55fc276004529af47c53dbb9f2204` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:21.946439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b105d18-d5bb-4fd9-9d2f-f5ecb661c908 None admin] RBAC: Authorizing `identity:get_project(project_id=f0a55fc276004529af47c53dbb9f2204)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:21.947900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b105d18-d5bb-4fd9-9d2f-f5ecb661c908 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:21.949495 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 580/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:55:21 2022] GET /identity/v3/projects/f0a55fc276004529af47c53dbb9f2204 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:21.955618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc21fc-5271-4bb8-b3d6-775d825d3bca None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:21.972409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:22.050712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc21fc-5271-4bb8-b3d6-775d825d3bca None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:22.083113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee88e3f-2bba-429a-812f-e069b65bffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:22.083699 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee88e3f-2bba-429a-812f-e069b65bffab None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:22.083971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee88e3f-2bba-429a-812f-e069b65bffab None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:22.084236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee88e3f-2bba-429a-812f-e069b65bffab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:22.084986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee88e3f-2bba-429a-812f-e069b65bffab None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=f0a55fc276004529af47c53dbb9f2204, user.id=6069ff6f26f443cbad983ad80c04b297) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:55:22.085288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee88e3f-2bba-429a-812f-e069b65bffab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:22.086758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee88e3f-2bba-429a-812f-e069b65bffab None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:22.092175 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 605/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:55:21 2022] GET /identity/v3/role_assignments?user.id=6069ff6f26f443cbad983ad80c04b297&scope.project.id=f0a55fc276004529af47c53dbb9f2204&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:24.220790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043bb2b-f22b-46c1-a3b1-e80a70792260 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:24.221325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043bb2b-f22b-46c1-a3b1-e80a70792260 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:24.221325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043bb2b-f22b-46c1-a3b1-e80a70792260 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:24.222666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 581/1186] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:55:24.228437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa95e4d-f757-4515-a6b1-480e6089b487 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:24.228849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa95e4d-f757-4515-a6b1-480e6089b487 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:24.228982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa95e4d-f757-4515-a6b1-480e6089b487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:24.247834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8fa95e4d-f757-4515-a6b1-480e6089b487 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:55:24.249021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8fa95e4d-f757-4515-a6b1-480e6089b487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:55:24.315602 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 606/1187] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:55:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:55:24.584482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043bb2b-f22b-46c1-a3b1-e80a70792260 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:24.672824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:24.683454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043bb2b-f22b-46c1-a3b1-e80a70792260 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:24.716635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe73497-e2b7-4922-9b7f-630d1673d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:24.717247 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe73497-e2b7-4922-9b7f-630d1673d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:24.717525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe73497-e2b7-4922-9b7f-630d1673d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:24.717796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe73497-e2b7-4922-9b7f-630d1673d9a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:24.727600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe73497-e2b7-4922-9b7f-630d1673d9a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-cfe73497-e2b7-4922-9b7f-630d1673d9a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:55:24.732390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:55:24.733470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:55:24.734642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 582/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:55:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:55:24.739600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa95e4d-f757-4515-a6b1-480e6089b487 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:24.751793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:24.841499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa95e4d-f757-4515-a6b1-480e6089b487 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:24.873816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5ffea-455b-4e40-90f1-274df4325595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:24.874417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5ffea-455b-4e40-90f1-274df4325595 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:24.874697 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5ffea-455b-4e40-90f1-274df4325595 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:24.874995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5ffea-455b-4e40-90f1-274df4325595 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:24.875652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5ffea-455b-4e40-90f1-274df4325595 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:55:24.875957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5ffea-455b-4e40-90f1-274df4325595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:24.877312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5ffea-455b-4e40-90f1-274df4325595 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:24.885859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 607/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:55:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:24.892125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe73497-e2b7-4922-9b7f-630d1673d9a9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:24.993833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe73497-e2b7-4922-9b7f-630d1673d9a9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:25.026520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69cb6d-020e-4dd2-b394-13bce7beed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:25.027166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69cb6d-020e-4dd2-b394-13bce7beed5d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:25.027449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69cb6d-020e-4dd2-b394-13bce7beed5d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:25.027722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69cb6d-020e-4dd2-b394-13bce7beed5d None admin] PATH_INFO: `/v3/users/6069ff6f26f443cbad983ad80c04b297` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:25.028838 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69cb6d-020e-4dd2-b394-13bce7beed5d None admin] RBAC: Authorizing `identity:get_user(user_id=6069ff6f26f443cbad983ad80c04b297)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:25.030260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69cb6d-020e-4dd2-b394-13bce7beed5d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:25.031883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 583/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:55:24 2022] GET /identity/v3/users/6069ff6f26f443cbad983ad80c04b297 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:25.038445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5ffea-455b-4e40-90f1-274df4325595 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:25.107529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:25.138502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5ffea-455b-4e40-90f1-274df4325595 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:25.172185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1cbee-e503-44aa-8008-d49c51080735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:25.172803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1cbee-e503-44aa-8008-d49c51080735 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:25.173087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1cbee-e503-44aa-8008-d49c51080735 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:25.173628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1cbee-e503-44aa-8008-d49c51080735 None admin] PATH_INFO: `/v3/projects/f0a55fc276004529af47c53dbb9f2204` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:25.174741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1cbee-e503-44aa-8008-d49c51080735 None admin] RBAC: Authorizing `identity:get_project(project_id=f0a55fc276004529af47c53dbb9f2204)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:25.176193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1cbee-e503-44aa-8008-d49c51080735 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:25.177815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 608/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:55:25 2022] GET /identity/v3/projects/f0a55fc276004529af47c53dbb9f2204 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:25.184278 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69cb6d-020e-4dd2-b394-13bce7beed5d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:25.206163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:25.283686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69cb6d-020e-4dd2-b394-13bce7beed5d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:25.316748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5983477-e9bb-44b9-b102-03602af705d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:25.317394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983477-e9bb-44b9-b102-03602af705d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:25.317674 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983477-e9bb-44b9-b102-03602af705d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:25.317973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983477-e9bb-44b9-b102-03602af705d9 None admin] PATH_INFO: `/v3/projects/f0a55fc276004529af47c53dbb9f2204/users/6069ff6f26f443cbad983ad80c04b297/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:25.320519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5983477-e9bb-44b9-b102-03602af705d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0a55fc276004529af47c53dbb9f2204, user_id=6069ff6f26f443cbad983ad80c04b297, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:25.321939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5983477-e9bb-44b9-b102-03602af705d9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:25.334720 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 584/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:55:25 2022] PUT /identity/v3/projects/f0a55fc276004529af47c53dbb9f2204/users/6069ff6f26f443cbad983ad80c04b297/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:55:27.460185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bcfb6-3f5d-46ad-9fe5-45cdb8312098 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:27.460745 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bcfb6-3f5d-46ad-9fe5-45cdb8312098 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:27.460745 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bcfb6-3f5d-46ad-9fe5-45cdb8312098 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:27.462057 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 609/1193] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:55:27.467393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829804-1132-42d5-b219-d64872e735a6 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:27.467860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829804-1132-42d5-b219-d64872e735a6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:27.467967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829804-1132-42d5-b219-d64872e735a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:27.486383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a2829804-1132-42d5-b219-d64872e735a6 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:55:27.487603 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a2829804-1132-42d5-b219-d64872e735a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:55:27.555556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 585/1194] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:55:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:55:27.822796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bcfb6-3f5d-46ad-9fe5-45cdb8312098 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:27.922236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bcfb6-3f5d-46ad-9fe5-45cdb8312098 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:27.954924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e32618-726e-4ab3-91c7-493500861424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:27.955560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e32618-726e-4ab3-91c7-493500861424 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:27.955839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e32618-726e-4ab3-91c7-493500861424 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:27.956110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e32618-726e-4ab3-91c7-493500861424 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:27.964624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:27.968667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e32618-726e-4ab3-91c7-493500861424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-19e32618-726e-4ab3-91c7-493500861424 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:55:27.973694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:55:27.974809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:55:27.975815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 610/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:55:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:55:27.981117 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2829804-1132-42d5-b219-d64872e735a6 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:27.992658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:28.078991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2829804-1132-42d5-b219-d64872e735a6 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:28.112070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d2674-f93c-4b13-9c99-08aec60581c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:28.112662 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d2674-f93c-4b13-9c99-08aec60581c9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:28.112940 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d2674-f93c-4b13-9c99-08aec60581c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:28.113208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d2674-f93c-4b13-9c99-08aec60581c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:28.113858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d2674-f93c-4b13-9c99-08aec60581c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:55:28.114155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d2674-f93c-4b13-9c99-08aec60581c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:28.115534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d2674-f93c-4b13-9c99-08aec60581c9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:28.123338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 586/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:55:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:28.129425 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e32618-726e-4ab3-91c7-493500861424 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:28.228963 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e32618-726e-4ab3-91c7-493500861424 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:28.263315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e6484-3659-4a63-a0e4-7dea78207394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:28.263948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e6484-3659-4a63-a0e4-7dea78207394 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:28.264227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e6484-3659-4a63-a0e4-7dea78207394 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:28.264500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e6484-3659-4a63-a0e4-7dea78207394 None admin] PATH_INFO: `/v3/users/6069ff6f26f443cbad983ad80c04b297` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:28.265628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e6484-3659-4a63-a0e4-7dea78207394 None admin] RBAC: Authorizing `identity:get_user(user_id=6069ff6f26f443cbad983ad80c04b297)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:28.267080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e6484-3659-4a63-a0e4-7dea78207394 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:28.268669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 611/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 22 16:55:28 2022] GET /identity/v3/users/6069ff6f26f443cbad983ad80c04b297 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:28.274501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d2674-f93c-4b13-9c99-08aec60581c9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:28.343564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:28.376549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d2674-f93c-4b13-9c99-08aec60581c9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:28.409658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb403d-ad11-4e51-bde5-313bb3963633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:28.410264 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb403d-ad11-4e51-bde5-313bb3963633 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:28.410546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb403d-ad11-4e51-bde5-313bb3963633 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:28.410815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb403d-ad11-4e51-bde5-313bb3963633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:28.477135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb403d-ad11-4e51-bde5-313bb3963633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:28.478805 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb403d-ad11-4e51-bde5-313bb3963633 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:28.544760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:28.576968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 587/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:55:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:55:28.583164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e6484-3659-4a63-a0e4-7dea78207394 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:28.647249 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:28.680687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e6484-3659-4a63-a0e4-7dea78207394 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:28.716195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd7dfb-2f0c-464a-8c69-c3bae17a933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:28.716827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd7dfb-2f0c-464a-8c69-c3bae17a933e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:28.717099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd7dfb-2f0c-464a-8c69-c3bae17a933e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:28.717369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd7dfb-2f0c-464a-8c69-c3bae17a933e None admin] PATH_INFO: `/v3/projects/f0a55fc276004529af47c53dbb9f2204` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:28.718497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd7dfb-2f0c-464a-8c69-c3bae17a933e None admin] RBAC: Authorizing `identity:get_project(project_id=f0a55fc276004529af47c53dbb9f2204)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:28.719936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd7dfb-2f0c-464a-8c69-c3bae17a933e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:28.721570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 612/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 22 16:55:28 2022] GET /identity/v3/projects/f0a55fc276004529af47c53dbb9f2204 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:28.729024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb403d-ad11-4e51-bde5-313bb3963633 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:55:28.827179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb403d-ad11-4e51-bde5-313bb3963633 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:55:28.860973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb67ba4-5a3b-49b6-b894-650831f44540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:55:28.861563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb67ba4-5a3b-49b6-b894-650831f44540 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:55:28.861839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb67ba4-5a3b-49b6-b894-650831f44540 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:55:28.862108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb67ba4-5a3b-49b6-b894-650831f44540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:55:28.862824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb67ba4-5a3b-49b6-b894-650831f44540 None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=f0a55fc276004529af47c53dbb9f2204, user.id=6069ff6f26f443cbad983ad80c04b297) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:55:28.863159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb67ba4-5a3b-49b6-b894-650831f44540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:55:28.864597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb67ba4-5a3b-49b6-b894-650831f44540 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:55:28.870745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 588/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:55:28 2022] GET /identity/v3/role_assignments?user.id=6069ff6f26f443cbad983ad80c04b297&scope.project.id=f0a55fc276004529af47c53dbb9f2204&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:55:38.710124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=32 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:55:38.868696 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=82 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:25.534739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559bad1-04f4-4c38-add0-c4784537dad5 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:25.535966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559bad1-04f4-4c38-add0-c4784537dad5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:25.535966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559bad1-04f4-4c38-add0-c4784537dad5 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:25.537476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 613/1201] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:56:25.544685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ca2de-6a33-4dea-9a65-e83550c62b13 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:25.547012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ca2de-6a33-4dea-9a65-e83550c62b13 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:25.547012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ca2de-6a33-4dea-9a65-e83550c62b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:25.582387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e47ca2de-6a33-4dea-9a65-e83550c62b13 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:56:25.584046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e47ca2de-6a33-4dea-9a65-e83550c62b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:56:25.682812 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 589/1202] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:56:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:56:25.986250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559bad1-04f4-4c38-add0-c4784537dad5 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:26.132880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559bad1-04f4-4c38-add0-c4784537dad5 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:26.179922 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0e82-5563-4a76-9908-571d6c80c352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:26.180644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0e82-5563-4a76-9908-571d6c80c352 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:26.181007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0e82-5563-4a76-9908-571d6c80c352 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:26.181280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0e82-5563-4a76-9908-571d6c80c352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:26.194145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f0e82-5563-4a76-9908-571d6c80c352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:26.195674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f0e82-5563-4a76-9908-571d6c80c352 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-f84f0e82-5563-4a76-9908-571d6c80c352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:26.211236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:26.212337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:26.213327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:26.213327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:26.213327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:26.213327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 614/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:26.223335 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ca2de-6a33-4dea-9a65-e83550c62b13 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:26.359361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ca2de-6a33-4dea-9a65-e83550c62b13 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:26.396524 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:26.411297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0e0d0-3ac1-4145-bcbc-b87aad65ba33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:26.412048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0e0d0-3ac1-4145-bcbc-b87aad65ba33 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:26.412457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0e0d0-3ac1-4145-bcbc-b87aad65ba33 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:26.412922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0e0d0-3ac1-4145-bcbc-b87aad65ba33 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:26.413614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0e0d0-3ac1-4145-bcbc-b87aad65ba33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:26.413921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0e0d0-3ac1-4145-bcbc-b87aad65ba33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:26.415333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0e0d0-3ac1-4145-bcbc-b87aad65ba33 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:26.434999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 590/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:56:26.437407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0e82-5563-4a76-9908-571d6c80c352 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:26.515481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:26.578256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0e82-5563-4a76-9908-571d6c80c352 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:26.621294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e37ce7-be3a-4a4c-8a48-f27e4030b3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:26.621850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e37ce7-be3a-4a4c-8a48-f27e4030b3ab None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:26.622185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e37ce7-be3a-4a4c-8a48-f27e4030b3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:26.622566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e37ce7-be3a-4a4c-8a48-f27e4030b3ab None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:26.623955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e37ce7-be3a-4a4c-8a48-f27e4030b3ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:26.625246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e37ce7-be3a-4a4c-8a48-f27e4030b3ab None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:26.662975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 615/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 22 16:56:26 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:56:29.215673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cd9a4-821c-43d3-9aa9-56fb8ed9010f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:29.216320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cd9a4-821c-43d3-9aa9-56fb8ed9010f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:29.216320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cd9a4-821c-43d3-9aa9-56fb8ed9010f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:29.217751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 591/1206] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:56:29.225147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5e2c2-d186-4bf1-9e18-3729f81b8295 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:29.225889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5e2c2-d186-4bf1-9e18-3729f81b8295 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:29.225889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5e2c2-d186-4bf1-9e18-3729f81b8295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:29.247587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-08e5e2c2-d186-4bf1-9e18-3729f81b8295 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:56:29.248823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-08e5e2c2-d186-4bf1-9e18-3729f81b8295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:56:29.326887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 616/1207] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:56:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:56:29.650694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cd9a4-821c-43d3-9aa9-56fb8ed9010f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:29.814874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cd9a4-821c-43d3-9aa9-56fb8ed9010f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:29.872695 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bca63-580d-4b78-9732-4dad9216d4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:29.873977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bca63-580d-4b78-9732-4dad9216d4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:29.874547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bca63-580d-4b78-9732-4dad9216d4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:29.875149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bca63-580d-4b78-9732-4dad9216d4f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:29.898144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bca63-580d-4b78-9732-4dad9216d4f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-188bca63-580d-4b78-9732-4dad9216d4f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:56:29.910514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:29.911883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:56:29.912909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:29.913588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 592/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:56:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:29.919056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5e2c2-d186-4bf1-9e18-3729f81b8295 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:29.964863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:30.041515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5e2c2-d186-4bf1-9e18-3729f81b8295 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:30.082312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716ed2d-d75c-459c-b245-d2b17b616294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:30.082920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716ed2d-d75c-459c-b245-d2b17b616294 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:30.083235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716ed2d-d75c-459c-b245-d2b17b616294 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:30.083509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716ed2d-d75c-459c-b245-d2b17b616294 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:30.084203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716ed2d-d75c-459c-b245-d2b17b616294 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:30.084483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716ed2d-d75c-459c-b245-d2b17b616294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:30.085846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716ed2d-d75c-459c-b245-d2b17b616294 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:30.094976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 617/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:56:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:30.101054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bca63-580d-4b78-9732-4dad9216d4f6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:30.158436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:30.222346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bca63-580d-4b78-9732-4dad9216d4f6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:30.263699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca43fc8-0ca9-476e-839b-5d822ce7c3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:30.264339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca43fc8-0ca9-476e-839b-5d822ce7c3fe None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:30.264618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca43fc8-0ca9-476e-839b-5d822ce7c3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:30.264892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca43fc8-0ca9-476e-839b-5d822ce7c3fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:30.277817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca43fc8-0ca9-476e-839b-5d822ce7c3fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:30.280052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca43fc8-0ca9-476e-839b-5d822ce7c3fe None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-9ca43fc8-0ca9-476e-839b-5d822ce7c3fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:30.296616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:30.298159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:30.305199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:30.305199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:30.305199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:30.305199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 593/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:30.305445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716ed2d-d75c-459c-b245-d2b17b616294 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:30.415019 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:30.419284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716ed2d-d75c-459c-b245-d2b17b616294 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:30.462160 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade3836-b791-43a2-8fdc-3c1e04b71d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:30.462691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade3836-b791-43a2-8fdc-3c1e04b71d6e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:30.462989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade3836-b791-43a2-8fdc-3c1e04b71d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:30.463260 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade3836-b791-43a2-8fdc-3c1e04b71d6e None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:30.463911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade3836-b791-43a2-8fdc-3c1e04b71d6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:30.464235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade3836-b791-43a2-8fdc-3c1e04b71d6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:30.465573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade3836-b791-43a2-8fdc-3c1e04b71d6e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:30.483779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 618/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:30.485570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca43fc8-0ca9-476e-839b-5d822ce7c3fe None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:30.626297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca43fc8-0ca9-476e-839b-5d822ce7c3fe None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:30.628147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:30.677645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954f54-6a9b-42ec-8d4c-04f95aa90567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:30.678529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954f54-6a9b-42ec-8d4c-04f95aa90567 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:30.678812 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954f54-6a9b-42ec-8d4c-04f95aa90567 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:30.679183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954f54-6a9b-42ec-8d4c-04f95aa90567 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:30.688171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24954f54-6a9b-42ec-8d4c-04f95aa90567 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:30.690106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24954f54-6a9b-42ec-8d4c-04f95aa90567 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-24954f54-6a9b-42ec-8d4c-04f95aa90567 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:56:30.705006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:30.706397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:30.707497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:30.707497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:30.707497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:56:30.707497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:56:30.707497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:56:30.707497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:56:30.707497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:56:30.707497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:56:30.707497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:30.707497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:30.707497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 594/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:30.712573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade3836-b791-43a2-8fdc-3c1e04b71d6e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:30.855867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade3836-b791-43a2-8fdc-3c1e04b71d6e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:30.926450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c309d-2f40-4aa9-b9ce-287cef700b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:30.927217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c309d-2f40-4aa9-b9ce-287cef700b12 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:30.927564 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c309d-2f40-4aa9-b9ce-287cef700b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:30.927931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c309d-2f40-4aa9-b9ce-287cef700b12 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:30.928738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c309d-2f40-4aa9-b9ce-287cef700b12 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:30.929145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c309d-2f40-4aa9-b9ce-287cef700b12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:30.930663 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c309d-2f40-4aa9-b9ce-287cef700b12 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:30.950738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 619/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 22 16:56:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:30.956569 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954f54-6a9b-42ec-8d4c-04f95aa90567 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:31.078570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954f54-6a9b-42ec-8d4c-04f95aa90567 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:31.117191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531e5a6-cb5e-425b-85e1-1eae8e156f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:31.117909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531e5a6-cb5e-425b-85e1-1eae8e156f56 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:31.118144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531e5a6-cb5e-425b-85e1-1eae8e156f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:31.118417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531e5a6-cb5e-425b-85e1-1eae8e156f56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:31.154599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:31.220829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531e5a6-cb5e-425b-85e1-1eae8e156f56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:31.222542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531e5a6-cb5e-425b-85e1-1eae8e156f56 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:31.363549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 595/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:56:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:56:31.369118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c309d-2f40-4aa9-b9ce-287cef700b12 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:31.390596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:31.515488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c309d-2f40-4aa9-b9ce-287cef700b12 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:31.571561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c39b58-e8fb-46d2-a028-2afc45fbb3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:31.572143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c39b58-e8fb-46d2-a028-2afc45fbb3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:31.572420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c39b58-e8fb-46d2-a028-2afc45fbb3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:31.572696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c39b58-e8fb-46d2-a028-2afc45fbb3a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:31.588788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c39b58-e8fb-46d2-a028-2afc45fbb3a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:31.590269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c39b58-e8fb-46d2-a028-2afc45fbb3a2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-97c39b58-e8fb-46d2-a028-2afc45fbb3a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:31.609619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:31.610760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:31.611731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:31.611731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:31.611731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:31.611731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 620/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:56:31.617285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531e5a6-cb5e-425b-85e1-1eae8e156f56 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:31.714413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:31.776571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531e5a6-cb5e-425b-85e1-1eae8e156f56 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:31.822133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009ddf2-bbcd-44bc-ba50-49885e99a6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:31.822752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009ddf2-bbcd-44bc-ba50-49885e99a6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:31.823059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009ddf2-bbcd-44bc-ba50-49885e99a6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:31.823328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009ddf2-bbcd-44bc-ba50-49885e99a6e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:31.823972 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009ddf2-bbcd-44bc-ba50-49885e99a6e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:31.824275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009ddf2-bbcd-44bc-ba50-49885e99a6e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:31.825697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009ddf2-bbcd-44bc-ba50-49885e99a6e6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:31.841591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 596/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:31.847902 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c39b58-e8fb-46d2-a028-2afc45fbb3a2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:31.935651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:31.978827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c39b58-e8fb-46d2-a028-2afc45fbb3a2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:32.017652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b08726-329b-4638-b0fe-e007150f3643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:32.018546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b08726-329b-4638-b0fe-e007150f3643 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:32.018832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b08726-329b-4638-b0fe-e007150f3643 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:32.019946 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b08726-329b-4638-b0fe-e007150f3643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:32.028806 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b08726-329b-4638-b0fe-e007150f3643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:32.030272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b08726-329b-4638-b0fe-e007150f3643 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-93b08726-329b-4638-b0fe-e007150f3643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:56:32.043821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:32.045015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:32.046112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:56:32.046112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:56:32.046112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:56:32.046112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:56:32.046112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:56:32.046112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:56:32.046112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:32.046112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:32.046112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 621/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:56:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:32.051516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009ddf2-bbcd-44bc-ba50-49885e99a6e6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:32.171468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009ddf2-bbcd-44bc-ba50-49885e99a6e6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:32.198098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:32.212416 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1904de-cb31-4271-90a1-30297dcf78d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:32.213097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1904de-cb31-4271-90a1-30297dcf78d6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:32.213398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1904de-cb31-4271-90a1-30297dcf78d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:32.213673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1904de-cb31-4271-90a1-30297dcf78d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:32.214421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1904de-cb31-4271-90a1-30297dcf78d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:32.214679 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1904de-cb31-4271-90a1-30297dcf78d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:32.216249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1904de-cb31-4271-90a1-30297dcf78d6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:32.230888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 597/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:56:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:32.237879 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b08726-329b-4638-b0fe-e007150f3643 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:32.365176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b08726-329b-4638-b0fe-e007150f3643 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:32.399647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:32.407111 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e68469-f4b4-47a8-9508-0232156902ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:32.408367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68469-f4b4-47a8-9508-0232156902ff None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:32.408782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68469-f4b4-47a8-9508-0232156902ff None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:32.409279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68469-f4b4-47a8-9508-0232156902ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:32.411304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e68469-f4b4-47a8-9508-0232156902ff None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=81088a327fd448ad8fec555f3d914be8) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:32.411953 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e68469-f4b4-47a8-9508-0232156902ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:32.416188 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e68469-f4b4-47a8-9508-0232156902ff None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:32.426797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 622/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:56:32 2022] GET /identity/v3/role_assignments?user.id=81088a327fd448ad8fec555f3d914be8&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:34.681717 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504cdd4-7015-40d6-9016-a0b4bcd0caee None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:34.682391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504cdd4-7015-40d6-9016-a0b4bcd0caee None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:34.682391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504cdd4-7015-40d6-9016-a0b4bcd0caee None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:34.683956 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 598/1220] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:56:34.694781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e27465-e65a-4a1e-9587-6f5c3bfffcff None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:34.694781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e27465-e65a-4a1e-9587-6f5c3bfffcff None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:34.694781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e27465-e65a-4a1e-9587-6f5c3bfffcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:34.712665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-57e27465-e65a-4a1e-9587-6f5c3bfffcff None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:56:34.713898 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-57e27465-e65a-4a1e-9587-6f5c3bfffcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:56:34.793398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 623/1221] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:56:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:56:35.082554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504cdd4-7015-40d6-9016-a0b4bcd0caee None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:35.198733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504cdd4-7015-40d6-9016-a0b4bcd0caee None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:35.240087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5858055-a5f0-448b-89f2-c80fc64acbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:35.240747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5858055-a5f0-448b-89f2-c80fc64acbec None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:35.241040 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5858055-a5f0-448b-89f2-c80fc64acbec None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:35.241314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5858055-a5f0-448b-89f2-c80fc64acbec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:35.254220 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5858055-a5f0-448b-89f2-c80fc64acbec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-b5858055-a5f0-448b-89f2-c80fc64acbec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:56:35.259226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:35.260455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:35.261370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 599/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:56:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:35.266699 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e27465-e65a-4a1e-9587-6f5c3bfffcff None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:35.373034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e27465-e65a-4a1e-9587-6f5c3bfffcff None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:35.397834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:35.412110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598401d9-e933-457c-8332-c7124c596e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:35.412754 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598401d9-e933-457c-8332-c7124c596e0a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:35.413113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598401d9-e933-457c-8332-c7124c596e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:35.413460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598401d9-e933-457c-8332-c7124c596e0a None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:35.414158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598401d9-e933-457c-8332-c7124c596e0a None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:35.414652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598401d9-e933-457c-8332-c7124c596e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:35.416565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598401d9-e933-457c-8332-c7124c596e0a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:35.428199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 624/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:56:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:35.434786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5858055-a5f0-448b-89f2-c80fc64acbec None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:35.483359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:35.544341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5858055-a5f0-448b-89f2-c80fc64acbec None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:35.588488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec6e7b-fbd5-43f1-95bd-6c897167ffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:35.589248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6e7b-fbd5-43f1-95bd-6c897167ffd7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:35.589530 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6e7b-fbd5-43f1-95bd-6c897167ffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:35.589880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6e7b-fbd5-43f1-95bd-6c897167ffd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:35.601388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec6e7b-fbd5-43f1-95bd-6c897167ffd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:35.602883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec6e7b-fbd5-43f1-95bd-6c897167ffd7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-6eec6e7b-fbd5-43f1-95bd-6c897167ffd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:35.619650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:35.620931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:35.622142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:35.622142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:35.622142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:35.622142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 600/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:35.629192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598401d9-e933-457c-8332-c7124c596e0a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:35.746262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598401d9-e933-457c-8332-c7124c596e0a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:35.786743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c592c-0121-4b55-9d86-af6d6475f55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:35.787396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c592c-0121-4b55-9d86-af6d6475f55c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:35.787675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c592c-0121-4b55-9d86-af6d6475f55c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:35.787941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c592c-0121-4b55-9d86-af6d6475f55c None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:35.788577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c592c-0121-4b55-9d86-af6d6475f55c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:35.788875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c592c-0121-4b55-9d86-af6d6475f55c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:35.790236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c592c-0121-4b55-9d86-af6d6475f55c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:35.805837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 625/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:35.812599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec6e7b-fbd5-43f1-95bd-6c897167ffd7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:35.925506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:35.933176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec6e7b-fbd5-43f1-95bd-6c897167ffd7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:35.974907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688a52a-6b0e-4b6c-bf39-1ea1532f6b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:35.975588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688a52a-6b0e-4b6c-bf39-1ea1532f6b66 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:35.975871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688a52a-6b0e-4b6c-bf39-1ea1532f6b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:35.976152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688a52a-6b0e-4b6c-bf39-1ea1532f6b66 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:35.985600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e688a52a-6b0e-4b6c-bf39-1ea1532f6b66 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:35.987228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e688a52a-6b0e-4b6c-bf39-1ea1532f6b66 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-e688a52a-6b0e-4b6c-bf39-1ea1532f6b66 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:56:36.000126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:36.001308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:36.002891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:36.002891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:36.002891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:56:36.002891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:56:36.002891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:56:36.002891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:56:36.002891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:56:36.002891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:56:36.002891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:36.002891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:36.002891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 601/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:36.007130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c592c-0121-4b55-9d86-af6d6475f55c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:36.068233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:36.130009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c592c-0121-4b55-9d86-af6d6475f55c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:36.174431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3b91a-f752-4a9d-bc90-9048f757de29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:36.175205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3b91a-f752-4a9d-bc90-9048f757de29 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:36.175484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3b91a-f752-4a9d-bc90-9048f757de29 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:36.175936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3b91a-f752-4a9d-bc90-9048f757de29 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:36.176800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3b91a-f752-4a9d-bc90-9048f757de29 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:36.177204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3b91a-f752-4a9d-bc90-9048f757de29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:36.178994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3b91a-f752-4a9d-bc90-9048f757de29 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:36.202714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 626/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 22 16:56:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:36.209211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688a52a-6b0e-4b6c-bf39-1ea1532f6b66 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:36.328017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688a52a-6b0e-4b6c-bf39-1ea1532f6b66 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:36.365442 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23bb58-c218-402a-b162-a0168e0faef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:36.366082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23bb58-c218-402a-b162-a0168e0faef7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:36.366366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23bb58-c218-402a-b162-a0168e0faef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:36.366640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23bb58-c218-402a-b162-a0168e0faef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:36.380530 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23bb58-c218-402a-b162-a0168e0faef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:36.382014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23bb58-c218-402a-b162-a0168e0faef7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-ee23bb58-c218-402a-b162-a0168e0faef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:36.397368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:36.398697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:36.399943 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:36.399943 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:36.399943 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:36.399943 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 602/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:36.404800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3b91a-f752-4a9d-bc90-9048f757de29 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:36.516883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3b91a-f752-4a9d-bc90-9048f757de29 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:36.519375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:36.553490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6c46d-7d6c-47a9-a583-c76903770985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:36.554107 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6c46d-7d6c-47a9-a583-c76903770985 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:36.554362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6c46d-7d6c-47a9-a583-c76903770985 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:36.554628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6c46d-7d6c-47a9-a583-c76903770985 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:36.555331 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6c46d-7d6c-47a9-a583-c76903770985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:36.555594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6c46d-7d6c-47a9-a583-c76903770985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:36.556945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6c46d-7d6c-47a9-a583-c76903770985 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:36.571865 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 627/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:36.577812 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23bb58-c218-402a-b162-a0168e0faef7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:36.592402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:36.694206 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23bb58-c218-402a-b162-a0168e0faef7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:36.735769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f49bd7-1336-4951-bffe-b4afd7271966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:36.736487 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49bd7-1336-4951-bffe-b4afd7271966 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:36.736817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49bd7-1336-4951-bffe-b4afd7271966 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:36.737092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49bd7-1336-4951-bffe-b4afd7271966 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:36.744918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f49bd7-1336-4951-bffe-b4afd7271966 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:36.746377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f49bd7-1336-4951-bffe-b4afd7271966 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-88f49bd7-1336-4951-bffe-b4afd7271966 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:56:36.760145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:36.761344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:36.762273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:56:36.762273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:56:36.762273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:56:36.762273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:56:36.762273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:56:36.762273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:56:36.762273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:36.762273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:36.762273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 603/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:36.767333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6c46d-7d6c-47a9-a583-c76903770985 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:36.873355 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6c46d-7d6c-47a9-a583-c76903770985 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:36.913929 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b2f87-8b0a-420a-a301-5f6bfe8d6b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:36.914611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b2f87-8b0a-420a-a301-5f6bfe8d6b3e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:36.914915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b2f87-8b0a-420a-a301-5f6bfe8d6b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:36.915266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b2f87-8b0a-420a-a301-5f6bfe8d6b3e None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:36.916088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b2f87-8b0a-420a-a301-5f6bfe8d6b3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:36.916412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b2f87-8b0a-420a-a301-5f6bfe8d6b3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:36.918020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b2f87-8b0a-420a-a301-5f6bfe8d6b3e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:36.931951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 628/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:56:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:36.938477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f49bd7-1336-4951-bffe-b4afd7271966 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:37.011029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:37.050000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f49bd7-1336-4951-bffe-b4afd7271966 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:37.090608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d485e7c-1b52-4c84-a327-a5804bf399ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:37.091306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d485e7c-1b52-4c84-a327-a5804bf399ca None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:37.091586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d485e7c-1b52-4c84-a327-a5804bf399ca None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:37.091853 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d485e7c-1b52-4c84-a327-a5804bf399ca None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/81088a327fd448ad8fec555f3d914be8/roles/6ae0df226be44eccbaa70e0140db1c40` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:37.107921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d485e7c-1b52-4c84-a327-a5804bf399ca None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=81088a327fd448ad8fec555f3d914be8, role_id=6ae0df226be44eccbaa70e0140db1c40)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:37.109366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d485e7c-1b52-4c84-a327-a5804bf399ca None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:37.124655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 604/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:56:36 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/81088a327fd448ad8fec555f3d914be8/roles/6ae0df226be44eccbaa70e0140db1c40 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 16:56:39.481213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96300cc-258b-4a9b-9c76-352849317cca None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:39.481803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96300cc-258b-4a9b-9c76-352849317cca None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:39.481931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96300cc-258b-4a9b-9c76-352849317cca None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:39.483556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 629/1233] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:56:39.493719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85677cbf-811a-4451-87d5-84ea7a0e4ec1 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:39.493719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85677cbf-811a-4451-87d5-84ea7a0e4ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:39.493719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85677cbf-811a-4451-87d5-84ea7a0e4ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:39.512506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-85677cbf-811a-4451-87d5-84ea7a0e4ec1 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:56:39.514007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-85677cbf-811a-4451-87d5-84ea7a0e4ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:56:39.596065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 605/1234] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:56:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:56:39.888452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96300cc-258b-4a9b-9c76-352849317cca None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:39.917433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:40.009050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96300cc-258b-4a9b-9c76-352849317cca None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:40.048837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed0b02-194b-4d1d-b1d8-f351ab048aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:40.049549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed0b02-194b-4d1d-b1d8-f351ab048aeb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:40.050467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed0b02-194b-4d1d-b1d8-f351ab048aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:40.050792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed0b02-194b-4d1d-b1d8-f351ab048aeb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:40.068781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed0b02-194b-4d1d-b1d8-f351ab048aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-7eed0b02-194b-4d1d-b1d8-f351ab048aeb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:56:40.073930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:40.075114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:40.076184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 630/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 22 16:56:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:56:40.081895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85677cbf-811a-4451-87d5-84ea7a0e4ec1 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:40.135268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:40.199447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85677cbf-811a-4451-87d5-84ea7a0e4ec1 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:40.239468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb1d00-aae6-4384-b593-b4e9e8996465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:40.240006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb1d00-aae6-4384-b593-b4e9e8996465 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:40.240341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb1d00-aae6-4384-b593-b4e9e8996465 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:40.240651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb1d00-aae6-4384-b593-b4e9e8996465 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:40.241353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb1d00-aae6-4384-b593-b4e9e8996465 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:40.241706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb1d00-aae6-4384-b593-b4e9e8996465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:40.243224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb1d00-aae6-4384-b593-b4e9e8996465 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:40.254629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 606/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 22 16:56:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:40.261054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed0b02-194b-4d1d-b1d8-f351ab048aeb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:40.356947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:40.374501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed0b02-194b-4d1d-b1d8-f351ab048aeb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:40.420615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14502a46-58a3-4a0c-b2ef-d769dfc0dbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:40.421313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14502a46-58a3-4a0c-b2ef-d769dfc0dbad None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:40.421623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14502a46-58a3-4a0c-b2ef-d769dfc0dbad None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:40.421957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14502a46-58a3-4a0c-b2ef-d769dfc0dbad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:40.435367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14502a46-58a3-4a0c-b2ef-d769dfc0dbad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:40.436925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14502a46-58a3-4a0c-b2ef-d769dfc0dbad None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-14502a46-58a3-4a0c-b2ef-d769dfc0dbad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:40.454740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:40.455901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:40.457871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:40.457871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:40.457871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:40.457871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 631/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:40.462663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb1d00-aae6-4384-b593-b4e9e8996465 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:40.584770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb1d00-aae6-4384-b593-b4e9e8996465 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:40.595811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:40.625525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187eb8b-e720-4241-ad38-905158927b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:40.626131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187eb8b-e720-4241-ad38-905158927b28 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:40.626425 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187eb8b-e720-4241-ad38-905158927b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:40.626725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187eb8b-e720-4241-ad38-905158927b28 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:40.627439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187eb8b-e720-4241-ad38-905158927b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:40.627745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187eb8b-e720-4241-ad38-905158927b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:40.629742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187eb8b-e720-4241-ad38-905158927b28 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:40.643833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 607/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:40.649871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14502a46-58a3-4a0c-b2ef-d769dfc0dbad None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:40.763914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14502a46-58a3-4a0c-b2ef-d769dfc0dbad None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:40.806123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ccb3d-8c70-4f7e-a4d1-5e512283b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:40.807032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ccb3d-8c70-4f7e-a4d1-5e512283b569 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:40.807439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ccb3d-8c70-4f7e-a4d1-5e512283b569 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:40.807537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ccb3d-8c70-4f7e-a4d1-5e512283b569 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:40.811627 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:40.818422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ccb3d-8c70-4f7e-a4d1-5e512283b569 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:40.820611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ccb3d-8c70-4f7e-a4d1-5e512283b569 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-6d9ccb3d-8c70-4f7e-a4d1-5e512283b569 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:56:40.833893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:40.835326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:40.836614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:40.836614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:40.836614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:56:40.836614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:56:40.836614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:56:40.836614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:56:40.836614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:56:40.836614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:56:40.836614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:40.836614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:40.836614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 632/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:40.840938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187eb8b-e720-4241-ad38-905158927b28 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:40.954547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187eb8b-e720-4241-ad38-905158927b28 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:40.996270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61317af2-24e5-473c-8ee7-afb56690c4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:40.997093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61317af2-24e5-473c-8ee7-afb56690c4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:40.997378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61317af2-24e5-473c-8ee7-afb56690c4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:40.997688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61317af2-24e5-473c-8ee7-afb56690c4f5 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:40.998518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61317af2-24e5-473c-8ee7-afb56690c4f5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:40.998833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61317af2-24e5-473c-8ee7-afb56690c4f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:41.000708 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61317af2-24e5-473c-8ee7-afb56690c4f5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:41.021928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 608/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 22 16:56:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:41.028669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ccb3d-8c70-4f7e-a4d1-5e512283b569 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:41.151156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ccb3d-8c70-4f7e-a4d1-5e512283b569 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:41.189784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0103794-d526-41f1-a9eb-92765f1fc768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:41.190563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0103794-d526-41f1-a9eb-92765f1fc768 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:41.190889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0103794-d526-41f1-a9eb-92765f1fc768 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:41.191228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0103794-d526-41f1-a9eb-92765f1fc768 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:41.264467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:41.274481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0103794-d526-41f1-a9eb-92765f1fc768 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:41.276372 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0103794-d526-41f1-a9eb-92765f1fc768 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:41.387424 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 633/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:56:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 16:56:41.395280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61317af2-24e5-473c-8ee7-afb56690c4f5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:41.430848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:41.518103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61317af2-24e5-473c-8ee7-afb56690c4f5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:41.559781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbdd18-3d91-4218-aa98-9923859d5b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:41.560652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbdd18-3d91-4218-aa98-9923859d5b0f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:41.560969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbdd18-3d91-4218-aa98-9923859d5b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:41.561309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbdd18-3d91-4218-aa98-9923859d5b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:41.575151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbdd18-3d91-4218-aa98-9923859d5b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:41.576799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbdd18-3d91-4218-aa98-9923859d5b0f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-ddcbdd18-3d91-4218-aa98-9923859d5b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:41.594379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:41.596000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:41.596959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:41.596959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:41.596959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:41.596959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 609/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:41.601516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0103794-d526-41f1-a9eb-92765f1fc768 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:41.714835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:41.725475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0103794-d526-41f1-a9eb-92765f1fc768 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:41.769425 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a76fb-ef24-4a2e-afb0-eebc12f0773d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:41.770060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a76fb-ef24-4a2e-afb0-eebc12f0773d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:41.770373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a76fb-ef24-4a2e-afb0-eebc12f0773d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:41.770669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a76fb-ef24-4a2e-afb0-eebc12f0773d None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:41.771369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a76fb-ef24-4a2e-afb0-eebc12f0773d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:41.771700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a76fb-ef24-4a2e-afb0-eebc12f0773d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:41.773301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a76fb-ef24-4a2e-afb0-eebc12f0773d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:41.787734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 634/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:41.794281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbdd18-3d91-4218-aa98-9923859d5b0f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:41.891478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:41.917445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbdd18-3d91-4218-aa98-9923859d5b0f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:41.961547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcd797-40a2-4a8c-b8e9-bfcb77d11e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:41.962444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcd797-40a2-4a8c-b8e9-bfcb77d11e16 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:41.962771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcd797-40a2-4a8c-b8e9-bfcb77d11e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:41.963180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcd797-40a2-4a8c-b8e9-bfcb77d11e16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:41.971366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcd797-40a2-4a8c-b8e9-bfcb77d11e16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:41.972937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcd797-40a2-4a8c-b8e9-bfcb77d11e16 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-89dcd797-40a2-4a8c-b8e9-bfcb77d11e16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:56:41.985671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:41.986868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:41.987926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:56:41.987926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:56:41.987926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:56:41.987926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:56:41.987926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:56:41.987926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:56:41.987926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:41.987926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:41.987926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 610/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:56:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:41.993569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a76fb-ef24-4a2e-afb0-eebc12f0773d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:42.118593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a76fb-ef24-4a2e-afb0-eebc12f0773d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:42.157842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0eb47-4b33-43a7-b895-1906427018bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:42.158469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0eb47-4b33-43a7-b895-1906427018bd None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:42.158744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0eb47-4b33-43a7-b895-1906427018bd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:42.159042 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0eb47-4b33-43a7-b895-1906427018bd None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:42.159757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0eb47-4b33-43a7-b895-1906427018bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:42.160059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0eb47-4b33-43a7-b895-1906427018bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:42.161485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0eb47-4b33-43a7-b895-1906427018bd None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:42.176922 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 635/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:56:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:56:42.183653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcd797-40a2-4a8c-b8e9-bfcb77d11e16 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:42.307381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcd797-40a2-4a8c-b8e9-bfcb77d11e16 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:42.342581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570b23b-b603-4563-ad60-d7325446492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:42.343218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570b23b-b603-4563-ad60-d7325446492a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:42.343499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570b23b-b603-4563-ad60-d7325446492a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:42.343776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570b23b-b603-4563-ad60-d7325446492a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:42.344472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570b23b-b603-4563-ad60-d7325446492a None admin] RBAC: Adding query filter params (role.id=6ae0df226be44eccbaa70e0140db1c40, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=81088a327fd448ad8fec555f3d914be8) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:42.344773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570b23b-b603-4563-ad60-d7325446492a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:42.346199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570b23b-b603-4563-ad60-d7325446492a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:42.353268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 611/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:56:42 2022] GET /identity/v3/role_assignments?user.id=81088a327fd448ad8fec555f3d914be8&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=6ae0df226be44eccbaa70e0140db1c40 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:44.903383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f5e3b-d594-4d71-92c1-17da7291324b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:44.903914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f5e3b-d594-4d71-92c1-17da7291324b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:44.904046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f5e3b-d594-4d71-92c1-17da7291324b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:44.905795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 636/1247] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:56:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:56:44.912316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17496-dd67-49e7-84ba-b551440f146c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:44.912748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17496-dd67-49e7-84ba-b551440f146c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:44.912885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17496-dd67-49e7-84ba-b551440f146c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:44.919328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:44.932878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7fe17496-dd67-49e7-84ba-b551440f146c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:56:44.934051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7fe17496-dd67-49e7-84ba-b551440f146c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:56:45.005390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 612/1248] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:56:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:56:45.278387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f5e3b-d594-4d71-92c1-17da7291324b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:45.297318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:45.395187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f5e3b-d594-4d71-92c1-17da7291324b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:45.443067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e2f6e-5ff2-4c22-ac86-c3def174c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:45.444006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e2f6e-5ff2-4c22-ac86-c3def174c616 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:45.444283 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e2f6e-5ff2-4c22-ac86-c3def174c616 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:45.444543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e2f6e-5ff2-4c22-ac86-c3def174c616 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:45.457525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e2f6e-5ff2-4c22-ac86-c3def174c616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-9b8e2f6e-5ff2-4c22-ac86-c3def174c616 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:56:45.463659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:45.464649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:45.465530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 637/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:56:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:45.470830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe17496-dd67-49e7-84ba-b551440f146c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:45.583901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe17496-dd67-49e7-84ba-b551440f146c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:45.615419 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:45.621198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbfb33-768b-4cbf-ad07-5dc953ee300c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:45.621812 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbfb33-768b-4cbf-ad07-5dc953ee300c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:45.622091 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbfb33-768b-4cbf-ad07-5dc953ee300c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:45.622364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbfb33-768b-4cbf-ad07-5dc953ee300c None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:45.623058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbfb33-768b-4cbf-ad07-5dc953ee300c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:45.623366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbfb33-768b-4cbf-ad07-5dc953ee300c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:45.624741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbfb33-768b-4cbf-ad07-5dc953ee300c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:45.634320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 613/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:56:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:45.640251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e2f6e-5ff2-4c22-ac86-c3def174c616 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:45.736233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:45.762646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e2f6e-5ff2-4c22-ac86-c3def174c616 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:45.802872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6230f59-146d-47ca-9bb2-dbec88b22dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:45.803550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6230f59-146d-47ca-9bb2-dbec88b22dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:45.803963 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6230f59-146d-47ca-9bb2-dbec88b22dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:45.804424 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6230f59-146d-47ca-9bb2-dbec88b22dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:45.816130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6230f59-146d-47ca-9bb2-dbec88b22dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:45.817813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6230f59-146d-47ca-9bb2-dbec88b22dd2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a6230f59-146d-47ca-9bb2-dbec88b22dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:45.833639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:45.834796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:45.836212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:45.836212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:45.836212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:45.836212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 638/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:45.841350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbfb33-768b-4cbf-ad07-5dc953ee300c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:45.954493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbfb33-768b-4cbf-ad07-5dc953ee300c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:45.993358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e93b2b-3c05-4926-81ff-ba2c0ff7f723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:45.993992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93b2b-3c05-4926-81ff-ba2c0ff7f723 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:45.994263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93b2b-3c05-4926-81ff-ba2c0ff7f723 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:45.994571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93b2b-3c05-4926-81ff-ba2c0ff7f723 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:45.995314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e93b2b-3c05-4926-81ff-ba2c0ff7f723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:45.995622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e93b2b-3c05-4926-81ff-ba2c0ff7f723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:45.997127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e93b2b-3c05-4926-81ff-ba2c0ff7f723 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:46.010515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 614/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:46.016542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6230f59-146d-47ca-9bb2-dbec88b22dd2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:46.128138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6230f59-146d-47ca-9bb2-dbec88b22dd2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:46.167004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:46.167004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d219a58-6845-4187-94d8-b305c0830de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:46.167361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d219a58-6845-4187-94d8-b305c0830de8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:46.167462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d219a58-6845-4187-94d8-b305c0830de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:46.167691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d219a58-6845-4187-94d8-b305c0830de8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:46.177306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d219a58-6845-4187-94d8-b305c0830de8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:46.178917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d219a58-6845-4187-94d8-b305c0830de8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-2d219a58-6845-4187-94d8-b305c0830de8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:56:46.192061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:46.193171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:46.194571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:46.194571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:46.194571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:56:46.194571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:56:46.194571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:56:46.194571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:56:46.194571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:56:46.194571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:56:46.194571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:46.194571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:46.194571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 639/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:56:46.208377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e93b2b-3c05-4926-81ff-ba2c0ff7f723 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:46.270457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:46.319973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e93b2b-3c05-4926-81ff-ba2c0ff7f723 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:46.357710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e9d4d-7f51-479c-b1dc-cbc82c80f61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:46.358333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e9d4d-7f51-479c-b1dc-cbc82c80f61b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:46.358614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e9d4d-7f51-479c-b1dc-cbc82c80f61b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:46.358885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e9d4d-7f51-479c-b1dc-cbc82c80f61b None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:46.359667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e9d4d-7f51-479c-b1dc-cbc82c80f61b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:46.359971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e9d4d-7f51-479c-b1dc-cbc82c80f61b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:46.361453 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e9d4d-7f51-479c-b1dc-cbc82c80f61b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:46.381126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 615/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 22 16:56:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:46.387676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d219a58-6845-4187-94d8-b305c0830de8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:46.508470 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d219a58-6845-4187-94d8-b305c0830de8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:46.548534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68e7fb-44db-45e6-a5e4-3c09f05cedb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:46.550512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68e7fb-44db-45e6-a5e4-3c09f05cedb0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:46.550512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68e7fb-44db-45e6-a5e4-3c09f05cedb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:46.550512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68e7fb-44db-45e6-a5e4-3c09f05cedb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:46.622166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:46.638319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68e7fb-44db-45e6-a5e4-3c09f05cedb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:46.639787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68e7fb-44db-45e6-a5e4-3c09f05cedb0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:46.757809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 640/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:56:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 16:56:46.763995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e9d4d-7f51-479c-b1dc-cbc82c80f61b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:46.871195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e9d4d-7f51-479c-b1dc-cbc82c80f61b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:46.886468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:46.908354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80e25e-b472-492b-8862-c13d72f9b0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:46.908899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80e25e-b472-492b-8862-c13d72f9b0bf None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:46.909209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80e25e-b472-492b-8862-c13d72f9b0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:46.909476 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80e25e-b472-492b-8862-c13d72f9b0bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:46.919585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80e25e-b472-492b-8862-c13d72f9b0bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:46.921061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80e25e-b472-492b-8862-c13d72f9b0bf None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-6c80e25e-b472-492b-8862-c13d72f9b0bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:46.936251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:46.937491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:46.938588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:46.938588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:46.938588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:46.938588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 616/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:46.943818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68e7fb-44db-45e6-a5e4-3c09f05cedb0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:47.032271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:47.044756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68e7fb-44db-45e6-a5e4-3c09f05cedb0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:47.080269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f730f-0045-4ad1-a577-cc8cce72421c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:47.080948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f730f-0045-4ad1-a577-cc8cce72421c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:47.081720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f730f-0045-4ad1-a577-cc8cce72421c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:47.081995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f730f-0045-4ad1-a577-cc8cce72421c None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:47.082643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f730f-0045-4ad1-a577-cc8cce72421c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:47.082974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f730f-0045-4ad1-a577-cc8cce72421c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:47.084350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f730f-0045-4ad1-a577-cc8cce72421c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:47.096258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 641/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:47.101978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80e25e-b472-492b-8862-c13d72f9b0bf None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:47.207671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80e25e-b472-492b-8862-c13d72f9b0bf None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:47.242882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f4b63-39d8-40dd-a635-47611ec61aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:47.243874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f4b63-39d8-40dd-a635-47611ec61aae None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:47.244319 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f4b63-39d8-40dd-a635-47611ec61aae None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:47.244736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f4b63-39d8-40dd-a635-47611ec61aae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:47.251828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f4b63-39d8-40dd-a635-47611ec61aae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:47.253825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f4b63-39d8-40dd-a635-47611ec61aae None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-969f4b63-39d8-40dd-a635-47611ec61aae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:56:47.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:47.266576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:47.267555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:56:47.267555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:56:47.267555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:56:47.267555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:56:47.267555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:56:47.267555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:56:47.267555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:47.267555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:47.267555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 617/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:56:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:47.273284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f730f-0045-4ad1-a577-cc8cce72421c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:47.372392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f730f-0045-4ad1-a577-cc8cce72421c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:47.408566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3607a1-c89c-434e-88b9-3cb73fbbbeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:47.409091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3607a1-c89c-434e-88b9-3cb73fbbbeb3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:47.409367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3607a1-c89c-434e-88b9-3cb73fbbbeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:47.409639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3607a1-c89c-434e-88b9-3cb73fbbbeb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:47.410370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3607a1-c89c-434e-88b9-3cb73fbbbeb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:47.410677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3607a1-c89c-434e-88b9-3cb73fbbbeb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:47.412218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3607a1-c89c-434e-88b9-3cb73fbbbeb3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:47.426165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 642/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:56:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:47.432663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f4b63-39d8-40dd-a635-47611ec61aae None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:47.458563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:47.536977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f4b63-39d8-40dd-a635-47611ec61aae None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:47.580807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf21f80-d9a5-47ff-833a-b46e210092fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:47.581410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf21f80-d9a5-47ff-833a-b46e210092fa None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:47.581689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf21f80-d9a5-47ff-833a-b46e210092fa None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:47.581961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf21f80-d9a5-47ff-833a-b46e210092fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:47.582689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf21f80-d9a5-47ff-833a-b46e210092fa None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=81088a327fd448ad8fec555f3d914be8) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:47.583024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf21f80-d9a5-47ff-833a-b46e210092fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:47.584474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf21f80-d9a5-47ff-833a-b46e210092fa None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:47.590248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 618/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:56:47 2022] GET /identity/v3/role_assignments?user.id=81088a327fd448ad8fec555f3d914be8&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:49.757671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbcae5-9f2f-4cdf-8cc8-ff8c094534b4 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:49.758242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbcae5-9f2f-4cdf-8cc8-ff8c094534b4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:49.758242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbcae5-9f2f-4cdf-8cc8-ff8c094534b4 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:49.759585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 643/1261] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:56:49.765006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172cbf1-c8ef-4160-8449-54e5c5f26325 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:49.765412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172cbf1-c8ef-4160-8449-54e5c5f26325 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:49.765545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172cbf1-c8ef-4160-8449-54e5c5f26325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:49.784608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a172cbf1-c8ef-4160-8449-54e5c5f26325 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:56:49.785798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a172cbf1-c8ef-4160-8449-54e5c5f26325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:56:49.854058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 619/1262] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:56:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:56:50.116288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbcae5-9f2f-4cdf-8cc8-ff8c094534b4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:50.127237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:50.217875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbcae5-9f2f-4cdf-8cc8-ff8c094534b4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:50.253941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad38db-dad1-4b41-bb8f-06e53a6237af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:50.254568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad38db-dad1-4b41-bb8f-06e53a6237af None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:50.254850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad38db-dad1-4b41-bb8f-06e53a6237af None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:50.255147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad38db-dad1-4b41-bb8f-06e53a6237af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:50.265278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad38db-dad1-4b41-bb8f-06e53a6237af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-23ad38db-dad1-4b41-bb8f-06e53a6237af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:56:50.270729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:50.271840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:50.272727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 644/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:56:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:50.277914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172cbf1-c8ef-4160-8449-54e5c5f26325 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:50.296305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:50.376808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172cbf1-c8ef-4160-8449-54e5c5f26325 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:50.416796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3157f7-010e-417c-aa1b-69b1aa114a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:50.417572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3157f7-010e-417c-aa1b-69b1aa114a7a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:50.417911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3157f7-010e-417c-aa1b-69b1aa114a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:50.418214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3157f7-010e-417c-aa1b-69b1aa114a7a None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:50.419046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3157f7-010e-417c-aa1b-69b1aa114a7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:50.419406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3157f7-010e-417c-aa1b-69b1aa114a7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:50.420995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3157f7-010e-417c-aa1b-69b1aa114a7a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:50.432486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 620/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:56:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:50.438687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad38db-dad1-4b41-bb8f-06e53a6237af None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:50.513850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:50.545684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad38db-dad1-4b41-bb8f-06e53a6237af None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:50.579156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7ab9b-5428-4f60-b77f-dc17540b465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:50.579769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7ab9b-5428-4f60-b77f-dc17540b465a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:50.580043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7ab9b-5428-4f60-b77f-dc17540b465a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:50.580312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7ab9b-5428-4f60-b77f-dc17540b465a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:50.592643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a7ab9b-5428-4f60-b77f-dc17540b465a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:50.594411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a7ab9b-5428-4f60-b77f-dc17540b465a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-c6a7ab9b-5428-4f60-b77f-dc17540b465a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:50.608707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:50.609954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:50.611087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:50.611087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:50.611087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:50.611087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 645/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:56:50.616163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3157f7-010e-417c-aa1b-69b1aa114a7a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:50.697806 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:50.717829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3157f7-010e-417c-aa1b-69b1aa114a7a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:50.751320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd3e71-fe3e-414b-9129-9ca8134fd236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:50.751926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd3e71-fe3e-414b-9129-9ca8134fd236 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:50.752206 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd3e71-fe3e-414b-9129-9ca8134fd236 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:50.752472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd3e71-fe3e-414b-9129-9ca8134fd236 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:50.753113 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd3e71-fe3e-414b-9129-9ca8134fd236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:50.753411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd3e71-fe3e-414b-9129-9ca8134fd236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:50.754770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd3e71-fe3e-414b-9129-9ca8134fd236 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:50.766858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 621/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:50.772517 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7ab9b-5428-4f60-b77f-dc17540b465a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:50.871826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7ab9b-5428-4f60-b77f-dc17540b465a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:50.891148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:50.905592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da10af-92cf-4d24-9407-719acfbbb701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:50.906215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da10af-92cf-4d24-9407-719acfbbb701 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:50.906492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da10af-92cf-4d24-9407-719acfbbb701 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:50.906763 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da10af-92cf-4d24-9407-719acfbbb701 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:50.914609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da10af-92cf-4d24-9407-719acfbbb701 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:50.916077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da10af-92cf-4d24-9407-719acfbbb701 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-38da10af-92cf-4d24-9407-719acfbbb701 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:56:50.927841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:50.928802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:50.929818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:50.929818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:50.929818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:56:50.929818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:56:50.929818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:56:50.929818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:56:50.929818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:56:50.929818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:56:50.929818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:50.929818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:50.929818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 646/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:50.934875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd3e71-fe3e-414b-9129-9ca8134fd236 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:51.031835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd3e71-fe3e-414b-9129-9ca8134fd236 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:51.069580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dc1e0-62db-4fbd-8ca5-34ce63e00d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:51.070199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dc1e0-62db-4fbd-8ca5-34ce63e00d5f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:51.070481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dc1e0-62db-4fbd-8ca5-34ce63e00d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:51.070746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dc1e0-62db-4fbd-8ca5-34ce63e00d5f None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:51.071533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dc1e0-62db-4fbd-8ca5-34ce63e00d5f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:51.071839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dc1e0-62db-4fbd-8ca5-34ce63e00d5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:51.073324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dc1e0-62db-4fbd-8ca5-34ce63e00d5f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:51.083969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:51.093292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 622/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 22 16:56:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:51.099873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da10af-92cf-4d24-9407-719acfbbb701 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:51.202423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da10af-92cf-4d24-9407-719acfbbb701 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:51.237976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa6ce7-9b89-48a5-a023-84172f904f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:51.238614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa6ce7-9b89-48a5-a023-84172f904f13 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:51.238896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa6ce7-9b89-48a5-a023-84172f904f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:51.239200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa6ce7-9b89-48a5-a023-84172f904f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:51.250693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa6ce7-9b89-48a5-a023-84172f904f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:51.252205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa6ce7-9b89-48a5-a023-84172f904f13 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-8aaa6ce7-9b89-48a5-a023-84172f904f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:51.267416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:51.269166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:51.270294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:51.270294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:51.270294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:51.272913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 647/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:51.280367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dc1e0-62db-4fbd-8ca5-34ce63e00d5f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:51.385264 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dc1e0-62db-4fbd-8ca5-34ce63e00d5f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:51.426387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287cce6-a5ff-454e-83bd-3a95879a6723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:51.427012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287cce6-a5ff-454e-83bd-3a95879a6723 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:51.427291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287cce6-a5ff-454e-83bd-3a95879a6723 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:51.427560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287cce6-a5ff-454e-83bd-3a95879a6723 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:51.428198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287cce6-a5ff-454e-83bd-3a95879a6723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:51.428496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287cce6-a5ff-454e-83bd-3a95879a6723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:51.429852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287cce6-a5ff-454e-83bd-3a95879a6723 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:51.442247 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 623/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:51.447954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa6ce7-9b89-48a5-a023-84172f904f13 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:51.479683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:51.548542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa6ce7-9b89-48a5-a023-84172f904f13 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:51.581028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb1e81-7b67-4b35-8f54-1059a853da37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:51.581656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb1e81-7b67-4b35-8f54-1059a853da37 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:51.581935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb1e81-7b67-4b35-8f54-1059a853da37 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:51.582206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb1e81-7b67-4b35-8f54-1059a853da37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:51.588952 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb1e81-7b67-4b35-8f54-1059a853da37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:51.590454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb1e81-7b67-4b35-8f54-1059a853da37 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-dbdb1e81-7b67-4b35-8f54-1059a853da37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:56:51.602267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:51.603392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:51.604409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:56:51.604409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:56:51.604409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:56:51.604409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:56:51.604409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:56:51.604409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:56:51.604409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:51.604409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:51.604409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 648/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:56:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:51.609913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287cce6-a5ff-454e-83bd-3a95879a6723 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:51.668758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:51.714261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287cce6-a5ff-454e-83bd-3a95879a6723 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:51.747562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152b7f2-7f0f-4fda-ba8e-7de66e190f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:51.748163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152b7f2-7f0f-4fda-ba8e-7de66e190f46 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:51.748441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152b7f2-7f0f-4fda-ba8e-7de66e190f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:51.748791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152b7f2-7f0f-4fda-ba8e-7de66e190f46 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:51.749425 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152b7f2-7f0f-4fda-ba8e-7de66e190f46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:51.749742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152b7f2-7f0f-4fda-ba8e-7de66e190f46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:51.751186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152b7f2-7f0f-4fda-ba8e-7de66e190f46 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:51.763802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 624/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:56:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:51.769858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb1e81-7b67-4b35-8f54-1059a853da37 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:51.861675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:51.870296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb1e81-7b67-4b35-8f54-1059a853da37 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:51.903366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ff93a-aab9-47bc-b14d-03eca9931b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:51.904015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ff93a-aab9-47bc-b14d-03eca9931b8f None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:51.904293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ff93a-aab9-47bc-b14d-03eca9931b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:51.904566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ff93a-aab9-47bc-b14d-03eca9931b8f None admin] PATH_INFO: `/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/81088a327fd448ad8fec555f3d914be8/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:51.906432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ff93a-aab9-47bc-b14d-03eca9931b8f None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b73d7ceaff442fa3d7215afba1b5c1, user_id=81088a327fd448ad8fec555f3d914be8, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:51.908039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ff93a-aab9-47bc-b14d-03eca9931b8f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:51.921324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 649/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 22 16:56:51 2022] PUT /identity/v3/projects/b2b73d7ceaff442fa3d7215afba1b5c1/users/81088a327fd448ad8fec555f3d914be8/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 16:56:54.089386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437d0ea-667f-4492-9d83-ccf73d714830 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:54.089951 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437d0ea-667f-4492-9d83-ccf73d714830 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:54.089951 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437d0ea-667f-4492-9d83-ccf73d714830 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:54.091207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 625/1274] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:56:54.097164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ba3e9-8ebc-4431-9a79-ff3f43bcaa09 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:54.097802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ba3e9-8ebc-4431-9a79-ff3f43bcaa09 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:54.097802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ba3e9-8ebc-4431-9a79-ff3f43bcaa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:54.117234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-369ba3e9-8ebc-4431-9a79-ff3f43bcaa09 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:56:54.118512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-369ba3e9-8ebc-4431-9a79-ff3f43bcaa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:56:54.190496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 650/1275] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:56:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:56:54.482440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437d0ea-667f-4492-9d83-ccf73d714830 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:54.590446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437d0ea-667f-4492-9d83-ccf73d714830 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:54.613547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:54.635436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06db2e4-00aa-4013-aac3-8358fc1b1f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:54.636136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06db2e4-00aa-4013-aac3-8358fc1b1f63 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:54.636467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06db2e4-00aa-4013-aac3-8358fc1b1f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:54.636779 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06db2e4-00aa-4013-aac3-8358fc1b1f63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:54.649097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06db2e4-00aa-4013-aac3-8358fc1b1f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f06db2e4-00aa-4013-aac3-8358fc1b1f63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 16:56:54.653977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:54.655669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:54.656539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 626/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 22 16:56:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:54.664397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ba3e9-8ebc-4431-9a79-ff3f43bcaa09 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:54.754888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:54.778561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ba3e9-8ebc-4431-9a79-ff3f43bcaa09 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:54.817716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d64ee-8448-4dee-af2b-cff5a17a3e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:54.818394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d64ee-8448-4dee-af2b-cff5a17a3e34 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:54.818718 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d64ee-8448-4dee-af2b-cff5a17a3e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:54.819064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d64ee-8448-4dee-af2b-cff5a17a3e34 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:54.819792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d64ee-8448-4dee-af2b-cff5a17a3e34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:54.820139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d64ee-8448-4dee-af2b-cff5a17a3e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:54.821556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d64ee-8448-4dee-af2b-cff5a17a3e34 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:54.831662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 651/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 22 16:56:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:54.837847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06db2e4-00aa-4013-aac3-8358fc1b1f63 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:54.945483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06db2e4-00aa-4013-aac3-8358fc1b1f63 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:54.983907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9954c-72fc-4887-910f-f05a0a24adbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:54.984544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9954c-72fc-4887-910f-f05a0a24adbe None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:54.984908 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9954c-72fc-4887-910f-f05a0a24adbe None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:54.985231 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9954c-72fc-4887-910f-f05a0a24adbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:54.997880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9954c-72fc-4887-910f-f05a0a24adbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:54.999386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9954c-72fc-4887-910f-f05a0a24adbe None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-cfb9954c-72fc-4887-910f-f05a0a24adbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:55.014930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:55.016254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:55.017253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:55.017253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:55.017253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:55.017253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 627/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:56:55.025892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d64ee-8448-4dee-af2b-cff5a17a3e34 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:55.151530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d64ee-8448-4dee-af2b-cff5a17a3e34 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:55.188790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944225ff-2fbf-4259-9750-00635430105d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:55.189396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944225ff-2fbf-4259-9750-00635430105d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:55.189676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944225ff-2fbf-4259-9750-00635430105d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:55.189994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944225ff-2fbf-4259-9750-00635430105d None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:55.190634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944225ff-2fbf-4259-9750-00635430105d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:55.190933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944225ff-2fbf-4259-9750-00635430105d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:55.192328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944225ff-2fbf-4259-9750-00635430105d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:55.205170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 652/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:55.211252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9954c-72fc-4887-910f-f05a0a24adbe None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:55.243742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:55.323842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9954c-72fc-4887-910f-f05a0a24adbe None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:55.359255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2661b25-025f-45c9-9071-27910f5df287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:55.359889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2661b25-025f-45c9-9071-27910f5df287 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:55.360169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2661b25-025f-45c9-9071-27910f5df287 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:55.360442 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2661b25-025f-45c9-9071-27910f5df287 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:55.368561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2661b25-025f-45c9-9071-27910f5df287 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:55.370013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2661b25-025f-45c9-9071-27910f5df287 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-b2661b25-025f-45c9-9071-27910f5df287 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 16:56:55.382317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:55.384322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:55.386825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:55.386825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:55.386825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 16:56:55.386825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 16:56:55.386825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 16:56:55.386825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 16:56:55.386825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 16:56:55.386825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 16:56:55.386825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 16:56:55.386825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:55.386825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 628/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:56:55.392294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944225ff-2fbf-4259-9750-00635430105d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:55.398744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:55.509069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944225ff-2fbf-4259-9750-00635430105d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:55.546282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08fa08-62e9-4745-a5b9-9cbc087649a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:55.546900 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08fa08-62e9-4745-a5b9-9cbc087649a3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:55.547217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08fa08-62e9-4745-a5b9-9cbc087649a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:55.547749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08fa08-62e9-4745-a5b9-9cbc087649a3 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:55.548497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08fa08-62e9-4745-a5b9-9cbc087649a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:55.548805 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08fa08-62e9-4745-a5b9-9cbc087649a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:55.550492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08fa08-62e9-4745-a5b9-9cbc087649a3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:55.571182 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 653/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 22 16:56:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:56:55.577531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2661b25-025f-45c9-9071-27910f5df287 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:55.675699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:55.687048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2661b25-025f-45c9-9071-27910f5df287 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:55.723727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718cd98-6486-42e0-9820-b6bcbdb9125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:55.724340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718cd98-6486-42e0-9820-b6bcbdb9125b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:55.724621 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718cd98-6486-42e0-9820-b6bcbdb9125b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:55.724889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718cd98-6486-42e0-9820-b6bcbdb9125b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:55.799906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718cd98-6486-42e0-9820-b6bcbdb9125b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:55.801375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718cd98-6486-42e0-9820-b6bcbdb9125b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:55.896608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:55.907991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 629/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 22 16:56:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 16:56:55.917676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08fa08-62e9-4745-a5b9-9cbc087649a3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:55.990212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:56.031672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08fa08-62e9-4745-a5b9-9cbc087649a3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:56.072654 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c860b-85fd-4123-8831-e9dcc21e3d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:56.073351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c860b-85fd-4123-8831-e9dcc21e3d9e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:56.073676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c860b-85fd-4123-8831-e9dcc21e3d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:56.073992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c860b-85fd-4123-8831-e9dcc21e3d9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:56.085566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c860b-85fd-4123-8831-e9dcc21e3d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:56.087151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c860b-85fd-4123-8831-e9dcc21e3d9e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-df9c860b-85fd-4123-8831-e9dcc21e3d9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 16:56:56.102481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:56.104149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 16:56:56.105282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 16:56:56.105282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 16:56:56.105282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:56.105282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 654/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 16:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:56.112340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718cd98-6486-42e0-9820-b6bcbdb9125b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:56.217312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718cd98-6486-42e0-9820-b6bcbdb9125b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:56.256818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5417-4558-4758-936d-2f86fe90dfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:56.257846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5417-4558-4758-936d-2f86fe90dfdf None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:56.258336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5417-4558-4758-936d-2f86fe90dfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:56.258813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5417-4558-4758-936d-2f86fe90dfdf None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:56.259991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc5417-4558-4758-936d-2f86fe90dfdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:56.260520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc5417-4558-4758-936d-2f86fe90dfdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:56.263064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc5417-4558-4758-936d-2f86fe90dfdf None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:56.283139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 630/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 22 16:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:56.289485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c860b-85fd-4123-8831-e9dcc21e3d9e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:56.392314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c860b-85fd-4123-8831-e9dcc21e3d9e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:56.407218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:56.434637 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f69b8-165a-468f-b7f0-ca08cb290042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:56.435431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f69b8-165a-468f-b7f0-ca08cb290042 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:56.435818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f69b8-165a-468f-b7f0-ca08cb290042 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:56.436203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f69b8-165a-468f-b7f0-ca08cb290042 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:56.444477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f69b8-165a-468f-b7f0-ca08cb290042 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:56.445933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f69b8-165a-468f-b7f0-ca08cb290042 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-1e8f69b8-165a-468f-b7f0-ca08cb290042 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 16:56:56.458464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:56.460295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:56.461980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 16:56:56.461980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 16:56:56.461980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 16:56:56.461980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 16:56:56.461980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 16:56:56.461980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 16:56:56.461980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 16:56:56.461980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Sep 22 16:56:56.461980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 655/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 22 16:56:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:56:56.469768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5417-4558-4758-936d-2f86fe90dfdf None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:56.555631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:56.615074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5417-4558-4758-936d-2f86fe90dfdf None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:56.657554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc225c-3bed-4050-8ac7-7ad7a9d50704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:56.658172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc225c-3bed-4050-8ac7-7ad7a9d50704 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:56.658452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc225c-3bed-4050-8ac7-7ad7a9d50704 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:56.659346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc225c-3bed-4050-8ac7-7ad7a9d50704 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:56.660053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc225c-3bed-4050-8ac7-7ad7a9d50704 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:56.660364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc225c-3bed-4050-8ac7-7ad7a9d50704 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:56.661801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc225c-3bed-4050-8ac7-7ad7a9d50704 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:56.676056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 631/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 22 16:56:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:56.682570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f69b8-165a-468f-b7f0-ca08cb290042 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:56.795156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f69b8-165a-468f-b7f0-ca08cb290042 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:56.831993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f9798-f660-4a20-9d73-e37f5a44ca32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:56.832604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f9798-f660-4a20-9d73-e37f5a44ca32 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:56.832886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f9798-f660-4a20-9d73-e37f5a44ca32 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:56.833159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f9798-f660-4a20-9d73-e37f5a44ca32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:56.833892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f9798-f660-4a20-9d73-e37f5a44ca32 None admin] RBAC: Adding query filter params (role.id=1aa220d46e7949729d20b04e4e0c838d, scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1, user.id=81088a327fd448ad8fec555f3d914be8) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:56.834201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f9798-f660-4a20-9d73-e37f5a44ca32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:56.835783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f9798-f660-4a20-9d73-e37f5a44ca32 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:56.842183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 656/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 22 16:56:56 2022] GET /identity/v3/role_assignments?user.id=81088a327fd448ad8fec555f3d914be8&scope.project.id=b2b73d7ceaff442fa3d7215afba1b5c1&role.id=1aa220d46e7949729d20b04e4e0c838d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:56:59.196878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6a033-ca84-41bb-a49a-07db659fdf4b None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:59.197432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6a033-ca84-41bb-a49a-07db659fdf4b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:59.197551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6a033-ca84-41bb-a49a-07db659fdf4b None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:59.198902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 632/1288] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:56:59.204933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642efa9-7f48-4c10-80c4-bab0af894fcb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:59.205729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642efa9-7f48-4c10-80c4-bab0af894fcb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:59.205729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642efa9-7f48-4c10-80c4-bab0af894fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:59.224889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5642efa9-7f48-4c10-80c4-bab0af894fcb None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:56:59.226284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5642efa9-7f48-4c10-80c4-bab0af894fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:56:59.259243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:59.310640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 657/1289] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:56:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:56:59.579909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6a033-ca84-41bb-a49a-07db659fdf4b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:59.692366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6a033-ca84-41bb-a49a-07db659fdf4b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:59.705050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:56:59.734282 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3e315-fc4b-4b81-861e-6287839ca752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:59.734996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3e315-fc4b-4b81-861e-6287839ca752 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:59.735312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3e315-fc4b-4b81-861e-6287839ca752 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:59.735611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3e315-fc4b-4b81-861e-6287839ca752 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:59.736262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3e315-fc4b-4b81-861e-6287839ca752 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:59.738394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3e315-fc4b-4b81-861e-6287839ca752 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-e8f3e315-fc4b-4b81-861e-6287839ca752 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:56:59.749768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:56:59.750925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:56:59.751819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 633/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 22 16:56:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 16:56:59.758059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642efa9-7f48-4c10-80c4-bab0af894fcb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:56:59.880508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642efa9-7f48-4c10-80c4-bab0af894fcb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:56:59.923171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938da3b-39ae-4836-8d4c-afd43bdbda33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:56:59.923921 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938da3b-39ae-4836-8d4c-afd43bdbda33 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:56:59.924820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938da3b-39ae-4836-8d4c-afd43bdbda33 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:56:59.925121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938da3b-39ae-4836-8d4c-afd43bdbda33 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:56:59.925844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938da3b-39ae-4836-8d4c-afd43bdbda33 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:56:59.926176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938da3b-39ae-4836-8d4c-afd43bdbda33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:56:59.927761 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938da3b-39ae-4836-8d4c-afd43bdbda33 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:56:59.934032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 658/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 22 16:56:59 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:56:59.940431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3e315-fc4b-4b81-861e-6287839ca752 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:57:00.060462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3e315-fc4b-4b81-861e-6287839ca752 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:57:00.100097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c80261-facf-47fd-89d0-c838302e19a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:57:00.100704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c80261-facf-47fd-89d0-c838302e19a6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:00.100980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c80261-facf-47fd-89d0-c838302e19a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:00.101247 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c80261-facf-47fd-89d0-c838302e19a6 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:00.101891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c80261-facf-47fd-89d0-c838302e19a6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:57:00.102190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c80261-facf-47fd-89d0-c838302e19a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:57:00.103606 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c80261-facf-47fd-89d0-c838302e19a6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:57:00.108946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 634/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 22 16:56:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:57:02.443165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed7af2-be1b-490f-85a1-5282b0a959e3 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:02.443985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed7af2-be1b-490f-85a1-5282b0a959e3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:02.443985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed7af2-be1b-490f-85a1-5282b0a959e3 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:02.445085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 659/1293] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:57:02.451809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9909c5-19c8-421a-8274-0282e1914ca7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:02.452887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9909c5-19c8-421a-8274-0282e1914ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:02.453389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9909c5-19c8-421a-8274-0282e1914ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:02.480103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3d9909c5-19c8-421a-8274-0282e1914ca7 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:57:02.481691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3d9909c5-19c8-421a-8274-0282e1914ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:57:02.530361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:57:02.572021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 635/1294] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:57:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:57:02.861649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed7af2-be1b-490f-85a1-5282b0a959e3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:57:02.917082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:57:03.044323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed7af2-be1b-490f-85a1-5282b0a959e3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:57:03.085349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341bb2c-60a3-4476-a0d8-670fb99e58f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:57:03.085972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341bb2c-60a3-4476-a0d8-670fb99e58f9 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:03.086251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341bb2c-60a3-4476-a0d8-670fb99e58f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:03.086520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341bb2c-60a3-4476-a0d8-670fb99e58f9 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:03.087270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341bb2c-60a3-4476-a0d8-670fb99e58f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:57:03.088687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341bb2c-60a3-4476-a0d8-670fb99e58f9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:57:03.106129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 660/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 22 16:57:02 2022] POST /identity/v3/services => generated 246 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:57:05.411806 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec5f79-607d-4bc1-812a-c98a19e53808 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:05.412429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec5f79-607d-4bc1-812a-c98a19e53808 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:05.412756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec5f79-607d-4bc1-812a-c98a19e53808 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:05.414788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 636/1296] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:57:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:57:05.427142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ef32f-ebd2-4a24-8d94-dc07521b86bc None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:05.427957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ef32f-ebd2-4a24-8d94-dc07521b86bc None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:05.427957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ef32f-ebd2-4a24-8d94-dc07521b86bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:05.448097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b42ef32f-ebd2-4a24-8d94-dc07521b86bc None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:57:05.449395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b42ef32f-ebd2-4a24-8d94-dc07521b86bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:57:05.531257 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 661/1297] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:57:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 16:57:05.808498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec5f79-607d-4bc1-812a-c98a19e53808 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:57:05.925639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec5f79-607d-4bc1-812a-c98a19e53808 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:57:05.965064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5ee0a-f481-4540-9f8c-d255efccb7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:57:05.965780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ee0a-f481-4540-9f8c-d255efccb7af None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:05.966103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ee0a-f481-4540-9f8c-d255efccb7af None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:05.966420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ee0a-f481-4540-9f8c-d255efccb7af None admin] PATH_INFO: `/v3/services/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:05.967172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5ee0a-f481-4540-9f8c-d255efccb7af None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:57:05.968707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5ee0a-f481-4540-9f8c-d255efccb7af None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c4a5ee0a-f481-4540-9f8c-d255efccb7af None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:57:05.978842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:57:05.980365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:57:05.982225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 637/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 22 16:57:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:57:05.988438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ef32f-ebd2-4a24-8d94-dc07521b86bc None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:57:06.008245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:57:06.100313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ef32f-ebd2-4a24-8d94-dc07521b86bc None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:57:06.138451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ffb74-692d-4353-8d4c-e5326e87d512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:57:06.139113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ffb74-692d-4353-8d4c-e5326e87d512 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:06.139396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ffb74-692d-4353-8d4c-e5326e87d512 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:06.139668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ffb74-692d-4353-8d4c-e5326e87d512 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:06.140312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ffb74-692d-4353-8d4c-e5326e87d512 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:57:06.140617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ffb74-692d-4353-8d4c-e5326e87d512 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:57:06.142012 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ffb74-692d-4353-8d4c-e5326e87d512 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:57:06.148391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 662/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 22 16:57:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 16:57:06.154907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5ee0a-f481-4540-9f8c-d255efccb7af None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:57:06.189535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:57:06.265896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5ee0a-f481-4540-9f8c-d255efccb7af None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:57:06.303165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a758b38-0545-4cff-97b1-a41a96952fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:57:06.303780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a758b38-0545-4cff-97b1-a41a96952fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:06.304087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a758b38-0545-4cff-97b1-a41a96952fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:06.304355 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a758b38-0545-4cff-97b1-a41a96952fe4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:06.305063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a758b38-0545-4cff-97b1-a41a96952fe4 None admin] RBAC: Adding query filter params (interface=public, service_id=6f78949add9a44719d3c5d02f566ca22, region_id=RegionOne) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:57:06.305371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a758b38-0545-4cff-97b1-a41a96952fe4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:57:06.306758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a758b38-0545-4cff-97b1-a41a96952fe4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:57:06.312783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 638/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 22 16:57:06 2022] GET /identity/v3/endpoints?service_id=6f78949add9a44719d3c5d02f566ca22&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:57:06.322276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ffb74-692d-4353-8d4c-e5326e87d512 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:57:06.448004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:57:06.470297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ffb74-692d-4353-8d4c-e5326e87d512 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:57:06.519586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c5874-e2e9-4b2f-bce3-d8c58c01a3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:57:06.520644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c5874-e2e9-4b2f-bce3-d8c58c01a3cc None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:06.521133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c5874-e2e9-4b2f-bce3-d8c58c01a3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:06.521617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c5874-e2e9-4b2f-bce3-d8c58c01a3cc None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:06.522739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c5874-e2e9-4b2f-bce3-d8c58c01a3cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:57:06.525317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c5874-e2e9-4b2f-bce3-d8c58c01a3cc None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:57:06.539450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 663/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 22 16:57:06 2022] GET /identity/v3/services => generated 2191 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 16:57:08.795223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd20c50-9a1c-45a0-882b-681c73e875f8 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:08.795813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd20c50-9a1c-45a0-882b-681c73e875f8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:08.795813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd20c50-9a1c-45a0-882b-681c73e875f8 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:08.797093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 639/1302] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:57:08.803347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e1615-6ef9-4c06-a911-2e6b1c29316b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:08.803924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e1615-6ef9-4c06-a911-2e6b1c29316b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:08.803924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e1615-6ef9-4c06-a911-2e6b1c29316b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:08.824517 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a33e1615-6ef9-4c06-a911-2e6b1c29316b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:57:08.825761 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a33e1615-6ef9-4c06-a911-2e6b1c29316b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:57:08.901341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 664/1303] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:57:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 16:57:09.178486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd20c50-9a1c-45a0-882b-681c73e875f8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:57:09.314607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:57:09.323839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd20c50-9a1c-45a0-882b-681c73e875f8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:57:09.362749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c470b87-e286-40bb-8d52-892bde521958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:57:09.363478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c470b87-e286-40bb-8d52-892bde521958 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:09.363797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c470b87-e286-40bb-8d52-892bde521958 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:09.364109 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c470b87-e286-40bb-8d52-892bde521958 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:09.364751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c470b87-e286-40bb-8d52-892bde521958 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:57:09.366244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c470b87-e286-40bb-8d52-892bde521958 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-2c470b87-e286-40bb-8d52-892bde521958 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 16:57:09.375921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 16:57:09.378446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 22 16:57:09.379564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 640/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 22 16:57:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 16:57:09.384828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e1615-6ef9-4c06-a911-2e6b1c29316b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:57:09.477081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:57:09.520205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e1615-6ef9-4c06-a911-2e6b1c29316b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:57:09.580074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981660-d3fc-4f7c-8e71-5c2d5e4bb52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:57:09.580769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981660-d3fc-4f7c-8e71-5c2d5e4bb52f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:09.581098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981660-d3fc-4f7c-8e71-5c2d5e4bb52f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:09.581372 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981660-d3fc-4f7c-8e71-5c2d5e4bb52f None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:09.582020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c981660-d3fc-4f7c-8e71-5c2d5e4bb52f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 16:57:09.582327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c981660-d3fc-4f7c-8e71-5c2d5e4bb52f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:57:09.583805 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c981660-d3fc-4f7c-8e71-5c2d5e4bb52f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:57:09.589729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 665/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 22 16:57:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 16:57:09.599757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c470b87-e286-40bb-8d52-892bde521958 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:57:09.769076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c470b87-e286-40bb-8d52-892bde521958 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:57:09.826368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cf71d-76cc-4b51-9793-4c5e4287f948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:57:09.827146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cf71d-76cc-4b51-9793-4c5e4287f948 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:57:09.827290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cf71d-76cc-4b51-9793-4c5e4287f948 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:57:09.827550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cf71d-76cc-4b51-9793-4c5e4287f948 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:57:09.828248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cf71d-76cc-4b51-9793-4c5e4287f948 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:57:09.829693 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cf71d-76cc-4b51-9793-4c5e4287f948 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:57:09.852562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 641/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 16:57:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 16:57:19.586559 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=30 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:57:19.847805 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:24.700409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9783-96ae-4319-8f3b-9f1e31bd96a0 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:24.701810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9783-96ae-4319-8f3b-9f1e31bd96a0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:24.701810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9783-96ae-4319-8f3b-9f1e31bd96a0 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:24.707847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 666/1307] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:58:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:58:24.710344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08d767-e9d3-45d8-9875-f10d3a60258e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:24.710864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08d767-e9d3-45d8-9875-f10d3a60258e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:24.711519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08d767-e9d3-45d8-9875-f10d3a60258e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:24.934043 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-cb08d767-e9d3-45d8-9875-f10d3a60258e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:24.936643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cb08d767-e9d3-45d8-9875-f10d3a60258e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:25.143310 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 642/1308] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:58:25.453787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dace037-4506-4164-816f-5951a9c38d53 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:25.454052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dace037-4506-4164-816f-5951a9c38d53 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:25.454329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dace037-4506-4164-816f-5951a9c38d53 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:25.455741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 667/1309] 10.222.0.16 () {54 vars in 964 bytes} [Thu Sep 22 16:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:58:25.462265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf5708-f225-45ee-99f8-244ef78c5e1e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:25.465719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf5708-f225-45ee-99f8-244ef78c5e1e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:25.465719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf5708-f225-45ee-99f8-244ef78c5e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:25.585770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c2bf5708-f225-45ee-99f8-244ef78c5e1e None None] MFA Rules not processed for user `8c2ed1eb664348e384824f39ca90c950`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:25.587976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c2bf5708-f225-45ee-99f8-244ef78c5e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:25.625768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:25.721225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 643/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Sep 22 16:58:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:58:25.731789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dace037-4506-4164-816f-5951a9c38d53 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:58:25.733765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7dace037-4506-4164-816f-5951a9c38d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:25.981348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dace037-4506-4164-816f-5951a9c38d53 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:58:26.149732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a63a6-4b9e-4b1d-9b71-4a1e7bed8089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:58:26.151595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a63a6-4b9e-4b1d-9b71-4a1e7bed8089 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:26.152189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a63a6-4b9e-4b1d-9b71-4a1e7bed8089 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:26.152781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a63a6-4b9e-4b1d-9b71-4a1e7bed8089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:26.156615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-649a63a6-4b9e-4b1d-9b71-4a1e7bed8089 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:26.278168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:26.399338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a63a6-4b9e-4b1d-9b71-4a1e7bed8089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:58:26.401375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a63a6-4b9e-4b1d-9b71-4a1e7bed8089 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:58:26.433865 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 668/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:58:29.507272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaef88f-4002-4cc7-9184-cb3fbda8c858 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:29.507824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaef88f-4002-4cc7-9184-cb3fbda8c858 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:29.507824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaef88f-4002-4cc7-9184-cb3fbda8c858 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:29.509152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 644/1312] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:58:29.515084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec010a0-0a53-4277-887a-ad8a60776561 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:29.515629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec010a0-0a53-4277-887a-ad8a60776561 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:29.515629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec010a0-0a53-4277-887a-ad8a60776561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:29.605085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-aec010a0-0a53-4277-887a-ad8a60776561 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:29.606371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-aec010a0-0a53-4277-887a-ad8a60776561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:29.722789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 669/1313] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:58:30.012239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaef88f-4002-4cc7-9184-cb3fbda8c858 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:58:30.014056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7eaef88f-4002-4cc7-9184-cb3fbda8c858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:30.156761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaef88f-4002-4cc7-9184-cb3fbda8c858 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:58:30.274841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:30.277716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59e956-c9b6-4c01-b1e6-ae185f89e82f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:58:30.278607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59e956-c9b6-4c01-b1e6-ae185f89e82f service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:30.279782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59e956-c9b6-4c01-b1e6-ae185f89e82f service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:30.280189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59e956-c9b6-4c01-b1e6-ae185f89e82f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:30.283592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bf59e956-c9b6-4c01-b1e6-ae185f89e82f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:30.431260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59e956-c9b6-4c01-b1e6-ae185f89e82f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:58:30.432725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59e956-c9b6-4c01-b1e6-ae185f89e82f service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:58:30.454603 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 645/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:58:33.148510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66ebed-a417-40c5-b400-891c5fcfb150 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:33.149087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66ebed-a417-40c5-b400-891c5fcfb150 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:33.149250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66ebed-a417-40c5-b400-891c5fcfb150 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:33.150689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 670/1315] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:58:33.157420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d7d5b-8cbd-4bf8-8c33-728d4aa32b8d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:33.158011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d7d5b-8cbd-4bf8-8c33-728d4aa32b8d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:33.158011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d7d5b-8cbd-4bf8-8c33-728d4aa32b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:33.237695 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-085d7d5b-8cbd-4bf8-8c33-728d4aa32b8d None None] MFA Rules not processed for user `0487c236de2145229f4d07ed5647f6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:33.238849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-085d7d5b-8cbd-4bf8-8c33-728d4aa32b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:33.299247 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:33.349841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 646/1316] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:58:33 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:58:33.662648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4134d-52ee-4b18-afda-6ed1ef8f2f66 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:33.662930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4134d-52ee-4b18-afda-6ed1ef8f2f66 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:33.663242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4134d-52ee-4b18-afda-6ed1ef8f2f66 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:33.664636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 671/1317] 10.222.0.16 () {54 vars in 964 bytes} [Thu Sep 22 16:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:58:33.672266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc331496-31c9-4b7f-a574-febd28b2f346 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:33.672548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc331496-31c9-4b7f-a574-febd28b2f346 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:33.672819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc331496-31c9-4b7f-a574-febd28b2f346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:33.761180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-cc331496-31c9-4b7f-a574-febd28b2f346 None None] MFA Rules not processed for user `8c2ed1eb664348e384824f39ca90c950`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:33.762409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cc331496-31c9-4b7f-a574-febd28b2f346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:33.847926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 647/1318] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Sep 22 16:58:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:58:33.857617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4134d-52ee-4b18-afda-6ed1ef8f2f66 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:58:33.859208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f2f4134d-52ee-4b18-afda-6ed1ef8f2f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:33.869181 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:33.971408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4134d-52ee-4b18-afda-6ed1ef8f2f66 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:58:34.052312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7acbb5-bb54-4ff5-bd9b-30dd5b7f2d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:58:34.052924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7acbb5-bb54-4ff5-bd9b-30dd5b7f2d8f service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:34.053199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7acbb5-bb54-4ff5-bd9b-30dd5b7f2d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:34.053468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7acbb5-bb54-4ff5-bd9b-30dd5b7f2d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:34.055154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3b7acbb5-bb54-4ff5-bd9b-30dd5b7f2d8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:34.121963 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:34.178735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7acbb5-bb54-4ff5-bd9b-30dd5b7f2d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:58:34.180224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7acbb5-bb54-4ff5-bd9b-30dd5b7f2d8f service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:58:34.199415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 672/1319] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:58:33 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:58:37.522065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05e018-00a8-47d1-a4f4-8a41f1b0bfcd None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:37.523081 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05e018-00a8-47d1-a4f4-8a41f1b0bfcd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:37.523081 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05e018-00a8-47d1-a4f4-8a41f1b0bfcd None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:37.524329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 648/1320] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:58:37.530822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ad446-c1b1-446d-87b7-eb0a1c161eaa None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:37.531543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ad446-c1b1-446d-87b7-eb0a1c161eaa None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:37.531543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ad446-c1b1-446d-87b7-eb0a1c161eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:37.613866 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-769ad446-c1b1-446d-87b7-eb0a1c161eaa None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:37.615231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-769ad446-c1b1-446d-87b7-eb0a1c161eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:37.725000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 673/1321] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:58:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:58:38.027003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05e018-00a8-47d1-a4f4-8a41f1b0bfcd None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:58:38.028570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cf05e018-00a8-47d1-a4f4-8a41f1b0bfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:38.099208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:38.162196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05e018-00a8-47d1-a4f4-8a41f1b0bfcd None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:58:38.249731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ba9e2-5fc6-4edd-b2b1-232894fab8eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:58:38.250344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ba9e2-5fc6-4edd-b2b1-232894fab8eb service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:38.250626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ba9e2-5fc6-4edd-b2b1-232894fab8eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:38.250899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ba9e2-5fc6-4edd-b2b1-232894fab8eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:38.252602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8a9ba9e2-5fc6-4edd-b2b1-232894fab8eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:38.382469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:38.398162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ba9e2-5fc6-4edd-b2b1-232894fab8eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:58:38.399649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ba9e2-5fc6-4edd-b2b1-232894fab8eb service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:58:38.420193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 649/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:58:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:58:41.531078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cca92-b216-4d32-8168-3378c61a3667 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:41.531948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cca92-b216-4d32-8168-3378c61a3667 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:41.531948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cca92-b216-4d32-8168-3378c61a3667 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:41.533250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 674/1323] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:58:41.538908 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e139ad-91dc-4bab-bd87-ec6faae85bc0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:41.539735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e139ad-91dc-4bab-bd87-ec6faae85bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:41.539735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e139ad-91dc-4bab-bd87-ec6faae85bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:41.622888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c4e139ad-91dc-4bab-bd87-ec6faae85bc0 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:41.624659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c4e139ad-91dc-4bab-bd87-ec6faae85bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:41.761013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 650/1324] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 16:58:42.050493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cca92-b216-4d32-8168-3378c61a3667 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:58:42.052168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f63cca92-b216-4d32-8168-3378c61a3667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:42.074542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:42.174106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cca92-b216-4d32-8168-3378c61a3667 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:58:42.258401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0f537-65b9-4446-94b0-d76e2dc28bde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:58:42.258872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0f537-65b9-4446-94b0-d76e2dc28bde service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:42.259247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0f537-65b9-4446-94b0-d76e2dc28bde service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:42.259516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0f537-65b9-4446-94b0-d76e2dc28bde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:42.261171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-81f0f537-65b9-4446-94b0-d76e2dc28bde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:42.348360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:42.399113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0f537-65b9-4446-94b0-d76e2dc28bde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:58:42.401096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0f537-65b9-4446-94b0-d76e2dc28bde service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:58:42.422211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 675/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:58:44.799326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16d964-247a-4978-9004-16ab832bf520 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:44.799918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16d964-247a-4978-9004-16ab832bf520 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:44.799918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16d964-247a-4978-9004-16ab832bf520 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:44.801254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 651/1326] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:58:44.808177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02180e-f8d4-4693-82e4-fabd80d4a7b4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:44.808751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02180e-f8d4-4693-82e4-fabd80d4a7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:44.809152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02180e-f8d4-4693-82e4-fabd80d4a7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:44.926313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2e02180e-f8d4-4693-82e4-fabd80d4a7b4 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:44.927606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2e02180e-f8d4-4693-82e4-fabd80d4a7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:45.070797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 676/1327] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:58:45.420049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16d964-247a-4978-9004-16ab832bf520 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:58:45.421893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1e16d964-247a-4978-9004-16ab832bf520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:45.536752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:45.603499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16d964-247a-4978-9004-16ab832bf520 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:58:45.698397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1af25d-e4ca-4bb8-8dda-4bfa0c05691e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:58:45.699178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1af25d-e4ca-4bb8-8dda-4bfa0c05691e service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:45.699527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1af25d-e4ca-4bb8-8dda-4bfa0c05691e service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:45.699875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1af25d-e4ca-4bb8-8dda-4bfa0c05691e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:45.702439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8a1af25d-e4ca-4bb8-8dda-4bfa0c05691e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:45.836673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:45.841427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1af25d-e4ca-4bb8-8dda-4bfa0c05691e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:58:45.843465 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1af25d-e4ca-4bb8-8dda-4bfa0c05691e service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:58:45.865597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 652/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:58:48.240337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2216-d3c6-4500-8b58-61ffef4a35b0 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:48.241199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2216-d3c6-4500-8b58-61ffef4a35b0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:48.241199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2216-d3c6-4500-8b58-61ffef4a35b0 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:48.242240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 677/1329] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:58:48.249721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bad5ed-a159-481c-b982-36256e5e5f7a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:48.250346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bad5ed-a159-481c-b982-36256e5e5f7a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:48.250712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bad5ed-a159-481c-b982-36256e5e5f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:48.377261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-84bad5ed-a159-481c-b982-36256e5e5f7a None None] MFA Rules not processed for user `0487c236de2145229f4d07ed5647f6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:48.378672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-84bad5ed-a159-481c-b982-36256e5e5f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:48.508268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 653/1330] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:58:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:58:48.817093 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb2216-d3c6-4500-8b58-61ffef4a35b0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:58:48.818742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3dbb2216-d3c6-4500-8b58-61ffef4a35b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:48.861779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:48.950729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb2216-d3c6-4500-8b58-61ffef4a35b0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:58:49.098381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb664f-de86-40f8-bb42-f26086ce2aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:58:49.099484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb664f-de86-40f8-bb42-f26086ce2aaa service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:49.099975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb664f-de86-40f8-bb42-f26086ce2aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:49.100372 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb664f-de86-40f8-bb42-f26086ce2aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:49.103729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5fdb664f-de86-40f8-bb42-f26086ce2aaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:49.271390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:49.331995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb664f-de86-40f8-bb42-f26086ce2aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:58:49.334802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb664f-de86-40f8-bb42-f26086ce2aaa service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:58:49.373051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 678/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:58:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:58:52.429835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf98d98-2323-429a-89f3-491d69ea0f29 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:52.430486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf98d98-2323-429a-89f3-491d69ea0f29 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:52.430486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf98d98-2323-429a-89f3-491d69ea0f29 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:52.431780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 654/1332] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:58:52.438420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ede7d7-32eb-4fc9-bd77-2db989cac044 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:52.439076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ede7d7-32eb-4fc9-bd77-2db989cac044 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:52.439076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ede7d7-32eb-4fc9-bd77-2db989cac044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:52.518131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-96ede7d7-32eb-4fc9-bd77-2db989cac044 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:52.519516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-96ede7d7-32eb-4fc9-bd77-2db989cac044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:52.636596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 679/1333] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:58:52.936987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf98d98-2323-429a-89f3-491d69ea0f29 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:58:52.938587 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bbf98d98-2323-429a-89f3-491d69ea0f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:53.037288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:53.078186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf98d98-2323-429a-89f3-491d69ea0f29 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:58:53.166911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d7041-fb81-4c58-bf8b-7773f3b82e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:58:53.167683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d7041-fb81-4c58-bf8b-7773f3b82e0b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:53.168097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d7041-fb81-4c58-bf8b-7773f3b82e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:53.168958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d7041-fb81-4c58-bf8b-7773f3b82e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:53.170881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2b3d7041-fb81-4c58-bf8b-7773f3b82e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:53.307223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d7041-fb81-4c58-bf8b-7773f3b82e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:58:53.308679 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d7041-fb81-4c58-bf8b-7773f3b82e0b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:58:53.320975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:53.332039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 655/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:58:55.685746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6455efa-9479-49d5-a96a-d0bd21660c83 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:55.686359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6455efa-9479-49d5-a96a-d0bd21660c83 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:55.686527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6455efa-9479-49d5-a96a-d0bd21660c83 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:55.688887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 680/1335] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:58:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:58:55.695198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a332c-34d2-43bb-ab66-9cfee08c8cb3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:55.696047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a332c-34d2-43bb-ab66-9cfee08c8cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:55.696047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a332c-34d2-43bb-ab66-9cfee08c8cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:55.777168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-277a332c-34d2-43bb-ab66-9cfee08c8cb3 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:55.778981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-277a332c-34d2-43bb-ab66-9cfee08c8cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:55.894558 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 656/1336] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:58:56.192418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6455efa-9479-49d5-a96a-d0bd21660c83 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:58:56.194045 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b6455efa-9479-49d5-a96a-d0bd21660c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:56.242411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:56.365625 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6455efa-9479-49d5-a96a-d0bd21660c83 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:58:56.500288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2fe47-9a57-4f9a-82f2-7ee08007a3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:58:56.500873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2fe47-9a57-4f9a-82f2-7ee08007a3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:56.501214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2fe47-9a57-4f9a-82f2-7ee08007a3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:56.501531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2fe47-9a57-4f9a-82f2-7ee08007a3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:56.503580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-80e2fe47-9a57-4f9a-82f2-7ee08007a3c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:56.605171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:58:56.635178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2fe47-9a57-4f9a-82f2-7ee08007a3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:58:56.636640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2fe47-9a57-4f9a-82f2-7ee08007a3c0 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:58:56.658057 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 681/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:58:59.398602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc6d52-86dd-42b8-990a-a2cca9de2b25 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:59.399386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc6d52-86dd-42b8-990a-a2cca9de2b25 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:59.399386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc6d52-86dd-42b8-990a-a2cca9de2b25 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:59.401212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 657/1338] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:58:59.408624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17599849-5385-4684-bd8a-310136781450 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:58:59.409216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17599849-5385-4684-bd8a-310136781450 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:58:59.409216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17599849-5385-4684-bd8a-310136781450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:58:59.491769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-17599849-5385-4684-bd8a-310136781450 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:58:59.492994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-17599849-5385-4684-bd8a-310136781450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:59.598658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 682/1339] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:58:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:58:59.887746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc6d52-86dd-42b8-990a-a2cca9de2b25 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:58:59.889437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-aabc6d52-86dd-42b8-990a-a2cca9de2b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:58:59.978308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:00.012859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc6d52-86dd-42b8-990a-a2cca9de2b25 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:59:00.099964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654155b-df15-445b-8e7a-e3e40143708b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:59:00.100482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654155b-df15-445b-8e7a-e3e40143708b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:00.100755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654155b-df15-445b-8e7a-e3e40143708b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:00.101016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654155b-df15-445b-8e7a-e3e40143708b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:00.102703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8654155b-df15-445b-8e7a-e3e40143708b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:00.239017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654155b-df15-445b-8e7a-e3e40143708b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:59:00.240624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654155b-df15-445b-8e7a-e3e40143708b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:59:00.263283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 658/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:58:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 16:59:05.568837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089adb20-42b6-4de1-ac00-26266cb1abf6 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:05.569416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089adb20-42b6-4de1-ac00-26266cb1abf6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:05.569829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089adb20-42b6-4de1-ac00-26266cb1abf6 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:05.572608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 683/1341] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:59:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:05.581887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938501d-805f-4743-9619-3a5105ddc873 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:05.582546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938501d-805f-4743-9619-3a5105ddc873 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:05.582857 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938501d-805f-4743-9619-3a5105ddc873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:05.592707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:05.713982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d938501d-805f-4743-9619-3a5105ddc873 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:05.713982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d938501d-805f-4743-9619-3a5105ddc873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:05.874640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 659/1342] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:06.167771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089adb20-42b6-4de1-ac00-26266cb1abf6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:59:06.169669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-089adb20-42b6-4de1-ac00-26266cb1abf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:06.231926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:06.306698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089adb20-42b6-4de1-ac00-26266cb1abf6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:59:06.414748 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a86e5-a516-4882-8c05-e9f334d909d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:59:06.415385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a86e5-a516-4882-8c05-e9f334d909d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:06.415662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a86e5-a516-4882-8c05-e9f334d909d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:06.415935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a86e5-a516-4882-8c05-e9f334d909d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:06.417638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e22a86e5-a516-4882-8c05-e9f334d909d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:06.531451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:06.552957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a86e5-a516-4882-8c05-e9f334d909d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:59:06.555758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a86e5-a516-4882-8c05-e9f334d909d0 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:59:06.583858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 684/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 16:59:09.766743 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003366b0-c571-4e7e-8299-3b7bf76bbf71 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:09.767298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003366b0-c571-4e7e-8299-3b7bf76bbf71 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:09.767298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003366b0-c571-4e7e-8299-3b7bf76bbf71 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:09.768494 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 660/1344] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:09.777905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2def8b-2500-4235-9101-8cb0130a8cb7 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:09.778569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2def8b-2500-4235-9101-8cb0130a8cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:09.778874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2def8b-2500-4235-9101-8cb0130a8cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:09.892546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ac2def8b-2500-4235-9101-8cb0130a8cb7 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:09.893936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ac2def8b-2500-4235-9101-8cb0130a8cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:10.014804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 685/1345] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:59:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:10.563234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003366b0-c571-4e7e-8299-3b7bf76bbf71 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:59:10.565024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-003366b0-c571-4e7e-8299-3b7bf76bbf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:10.697600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:10.726076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003366b0-c571-4e7e-8299-3b7bf76bbf71 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:59:10.847132 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f5443-a387-40df-a70e-2ce903892a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:59:10.848291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f5443-a387-40df-a70e-2ce903892a4d service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:10.848291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f5443-a387-40df-a70e-2ce903892a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:10.848291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f5443-a387-40df-a70e-2ce903892a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:10.850056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b10f5443-a387-40df-a70e-2ce903892a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:11.047616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f5443-a387-40df-a70e-2ce903892a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:59:11.050309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f5443-a387-40df-a70e-2ce903892a4d service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:59:11.084343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 661/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:59:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 16:59:16.259854 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e49e7c-2ce4-4511-933b-7d3966b42aab None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:16.260493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e49e7c-2ce4-4511-933b-7d3966b42aab None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:16.260636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e49e7c-2ce4-4511-933b-7d3966b42aab None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:16.263586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 686/1347] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:59:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:16.270494 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58bdd0-e474-4fc0-9c7c-32bafb93d632 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:16.271347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58bdd0-e474-4fc0-9c7c-32bafb93d632 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:16.271593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58bdd0-e474-4fc0-9c7c-32bafb93d632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:16.289289 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:16.363368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5d58bdd0-e474-4fc0-9c7c-32bafb93d632 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:16.364699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5d58bdd0-e474-4fc0-9c7c-32bafb93d632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:16.493454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 662/1348] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 16:59:16.794156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e49e7c-2ce4-4511-933b-7d3966b42aab None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:59:16.795769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c5e49e7c-2ce4-4511-933b-7d3966b42aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:16.865047 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:16.922764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e49e7c-2ce4-4511-933b-7d3966b42aab None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:59:17.020167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaba1ab-429f-4b7c-bcb9-4ad38732ea8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:59:17.020974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba1ab-429f-4b7c-bcb9-4ad38732ea8e service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:17.021467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba1ab-429f-4b7c-bcb9-4ad38732ea8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:17.021905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba1ab-429f-4b7c-bcb9-4ad38732ea8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:17.023896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-aaaba1ab-429f-4b7c-bcb9-4ad38732ea8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:17.211550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:17.218420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaba1ab-429f-4b7c-bcb9-4ad38732ea8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:59:17.220498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaba1ab-429f-4b7c-bcb9-4ad38732ea8e service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:59:17.254071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 687/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:59:23.443135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9b4fd-4404-4f80-a3dc-a8511b8e7fbb None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:23.443685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9b4fd-4404-4f80-a3dc-a8511b8e7fbb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:23.444020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9b4fd-4404-4f80-a3dc-a8511b8e7fbb None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:23.447089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 663/1350] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 16:59:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:23.453081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccf64d-7f9e-4beb-85cd-6d51b24fceb0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:23.453081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccf64d-7f9e-4beb-85cd-6d51b24fceb0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:23.453081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccf64d-7f9e-4beb-85cd-6d51b24fceb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:23.601062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a4ccf64d-7f9e-4beb-85cd-6d51b24fceb0 None None] MFA Rules not processed for user `81088a327fd448ad8fec555f3d914be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:23.604134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a4ccf64d-7f9e-4beb-85cd-6d51b24fceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:23.885511 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 688/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 16:59:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:26.498459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=59 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:29.150017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe282e1f-92b1-4a04-8c0c-507d6d29f273 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:29.151378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe282e1f-92b1-4a04-8c0c-507d6d29f273 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:29.151378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe282e1f-92b1-4a04-8c0c-507d6d29f273 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:29.152318 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 664/1352] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 16:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:59:29.157766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8feb70-dcd7-4f7d-a44a-a1c3148865c9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:29.158579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8feb70-dcd7-4f7d-a44a-a1c3148865c9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:29.158579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8feb70-dcd7-4f7d-a44a-a1c3148865c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:29.226527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:29.255364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ba8feb70-dcd7-4f7d-a44a-a1c3148865c9 None None] MFA Rules not processed for user `81088a327fd448ad8fec555f3d914be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:29.256774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ba8feb70-dcd7-4f7d-a44a-a1c3148865c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:29.384608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 689/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 16:59:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:30.978243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cda4a5-bedd-42a1-9cff-180d35160062 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:30.978907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cda4a5-bedd-42a1-9cff-180d35160062 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:30.979204 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cda4a5-bedd-42a1-9cff-180d35160062 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:30.981993 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 665/1354] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 16:59:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:59:30.989814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893cdf0-b113-461d-acb0-f1024a691005 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:30.990614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893cdf0-b113-461d-acb0-f1024a691005 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:30.990614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893cdf0-b113-461d-acb0-f1024a691005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:31.108414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5893cdf0-b113-461d-acb0-f1024a691005 None None] MFA Rules not processed for user `81088a327fd448ad8fec555f3d914be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:31.110481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5893cdf0-b113-461d-acb0-f1024a691005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:31.184508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:31.265422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 690/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 16:59:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:35.399269 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e2448-3e10-4408-8b5a-e4d1c85f2e1a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:35.399874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e2448-3e10-4408-8b5a-e4d1c85f2e1a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:35.399874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e2448-3e10-4408-8b5a-e4d1c85f2e1a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:35.401621 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 666/1356] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:35.409373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14417ed-3515-49ef-b013-5cf0d5ea531f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:35.409968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14417ed-3515-49ef-b013-5cf0d5ea531f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:35.410440 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14417ed-3515-49ef-b013-5cf0d5ea531f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:35.488771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f14417ed-3515-49ef-b013-5cf0d5ea531f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:35.490113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f14417ed-3515-49ef-b013-5cf0d5ea531f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:35.608104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 691/1357] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:36.053852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923638f9-3ab5-4ae9-9a75-a6b78de180d9 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:36.054326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923638f9-3ab5-4ae9-9a75-a6b78de180d9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:36.054788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923638f9-3ab5-4ae9-9a75-a6b78de180d9 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:36.057330 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 667/1358] 10.222.0.16 () {54 vars in 961 bytes} [Thu Sep 22 16:59:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:36.074148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcb5fb-d097-488f-b55e-4005ce61f7d1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:36.074605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcb5fb-d097-488f-b55e-4005ce61f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:36.075118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcb5fb-d097-488f-b55e-4005ce61f7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:36.103427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:36.205008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-fbfcb5fb-d097-488f-b55e-4005ce61f7d1 None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:36.206397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fbfcb5fb-d097-488f-b55e-4005ce61f7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:36.423911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 692/1359] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Sep 22 16:59:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 16:59:36.433010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923638f9-3ab5-4ae9-9a75-a6b78de180d9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:59:36.434536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-923638f9-3ab5-4ae9-9a75-a6b78de180d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:36.665860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923638f9-3ab5-4ae9-9a75-a6b78de180d9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:59:36.782359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:36.832562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a9585-caba-4bbf-881b-f704016a6e53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:59:36.833184 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a9585-caba-4bbf-881b-f704016a6e53 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:36.833467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a9585-caba-4bbf-881b-f704016a6e53 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:36.833740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a9585-caba-4bbf-881b-f704016a6e53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:36.835436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5a0a9585-caba-4bbf-881b-f704016a6e53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:36.954053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a9585-caba-4bbf-881b-f704016a6e53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:59:36.961202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a9585-caba-4bbf-881b-f704016a6e53 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:59:36.987637 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 668/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:59:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 16:59:39.935896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497cd74-362a-4e11-85ce-ba8c3956f65e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:39.936676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497cd74-362a-4e11-85ce-ba8c3956f65e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:39.936888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497cd74-362a-4e11-85ce-ba8c3956f65e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:39.939818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 693/1361] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:59:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:39.947121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36f89f-defc-4bef-99b7-6190bc52a2ef None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:39.948111 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36f89f-defc-4bef-99b7-6190bc52a2ef None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:39.948111 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36f89f-defc-4bef-99b7-6190bc52a2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:40.074657 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7e36f89f-defc-4bef-99b7-6190bc52a2ef None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:40.075823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7e36f89f-defc-4bef-99b7-6190bc52a2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:40.160495 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:40.291003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 669/1362] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 16:59:40.858069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497cd74-362a-4e11-85ce-ba8c3956f65e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:59:40.860938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e497cd74-362a-4e11-85ce-ba8c3956f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:40.989975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:41.127116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497cd74-362a-4e11-85ce-ba8c3956f65e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:59:41.277035 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0697cea-5244-4866-a2b7-124c9930010c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:59:41.277834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0697cea-5244-4866-a2b7-124c9930010c service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:41.278119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0697cea-5244-4866-a2b7-124c9930010c service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:41.278597 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0697cea-5244-4866-a2b7-124c9930010c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:41.280431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c0697cea-5244-4866-a2b7-124c9930010c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:41.338309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:41.472330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0697cea-5244-4866-a2b7-124c9930010c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:59:41.475492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0697cea-5244-4866-a2b7-124c9930010c service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:59:41.523181 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 694/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:59:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:59:44.846716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03260e68-9bb9-469b-8878-4d832ba32ad4 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:44.847351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03260e68-9bb9-469b-8878-4d832ba32ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:44.847725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03260e68-9bb9-469b-8878-4d832ba32ad4 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:44.850354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 670/1364] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:59:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:44.859819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf899004-add7-4392-a996-2bbb936a392d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:44.860409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf899004-add7-4392-a996-2bbb936a392d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:44.860690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf899004-add7-4392-a996-2bbb936a392d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:44.973677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bf899004-add7-4392-a996-2bbb936a392d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:44.975596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bf899004-add7-4392-a996-2bbb936a392d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:45.122827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:45.132762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 695/1365] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:45.428246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e3239-8ed7-417f-907d-dbf60c933fe4 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:45.428620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e3239-8ed7-417f-907d-dbf60c933fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:45.428802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e3239-8ed7-417f-907d-dbf60c933fe4 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:45.430449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 671/1366] 10.222.0.16 () {54 vars in 961 bytes} [Thu Sep 22 16:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:59:45.440704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66320a40-251e-4aad-bed6-ff5965eecbc1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:45.441077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66320a40-251e-4aad-bed6-ff5965eecbc1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:45.441470 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66320a40-251e-4aad-bed6-ff5965eecbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:45.548436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-66320a40-251e-4aad-bed6-ff5965eecbc1 None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:45.549802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-66320a40-251e-4aad-bed6-ff5965eecbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:45.708729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 696/1367] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Sep 22 16:59:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:45.728535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e3239-8ed7-417f-907d-dbf60c933fe4 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:59:45.730119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ae2e3239-8ed7-417f-907d-dbf60c933fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:45.929981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e3239-8ed7-417f-907d-dbf60c933fe4 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:59:45.960088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:46.097984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209b256-d803-42c3-b3a5-24497e812333 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:59:46.099341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209b256-d803-42c3-b3a5-24497e812333 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:46.099555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209b256-d803-42c3-b3a5-24497e812333 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:46.099859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209b256-d803-42c3-b3a5-24497e812333 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:46.101594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9209b256-d803-42c3-b3a5-24497e812333 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:46.234417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9209b256-d803-42c3-b3a5-24497e812333 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:59:46.236704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9209b256-d803-42c3-b3a5-24497e812333 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:59:46.263584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 672/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 16:59:47.424097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb21dc2-5df7-4d5f-9a29-794337944034 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:47.425224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb21dc2-5df7-4d5f-9a29-794337944034 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:47.425875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb21dc2-5df7-4d5f-9a29-794337944034 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:47.430040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 697/1369] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:59:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:47.440670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c961-504a-4cdd-a21c-e27ec9a8aa88 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:47.441301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c961-504a-4cdd-a21c-e27ec9a8aa88 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:47.441788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c961-504a-4cdd-a21c-e27ec9a8aa88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:47.512302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:47.599665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7ef8c961-504a-4cdd-a21c-e27ec9a8aa88 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:47.602121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7ef8c961-504a-4cdd-a21c-e27ec9a8aa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:47.855052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 673/1370] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:48.428344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fc668-a9a4-4cb6-8104-3aac6e4945e0 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:48.428940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fc668-a9a4-4cb6-8104-3aac6e4945e0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:48.429278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fc668-a9a4-4cb6-8104-3aac6e4945e0 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:48.431733 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 698/1371] 10.222.0.16 () {54 vars in 963 bytes} [Thu Sep 22 16:59:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:59:48.440924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c845c5-4fb1-4e63-812b-e933bfefa2a4 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:48.441791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c845c5-4fb1-4e63-812b-e933bfefa2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:48.441971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c845c5-4fb1-4e63-812b-e933bfefa2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:48.573046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-68c845c5-4fb1-4e63-812b-e933bfefa2a4 None None] MFA Rules not processed for user `ae1b8a51c9fa4f2594bdd79b58962b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:48.575178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-68c845c5-4fb1-4e63-812b-e933bfefa2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:48.643697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:48.687063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 674/1372] 10.222.0.16 () {58 vars in 1060 bytes} [Thu Sep 22 16:59:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:48.697020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fc668-a9a4-4cb6-8104-3aac6e4945e0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:59:48.699640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5e7fc668-a9a4-4cb6-8104-3aac6e4945e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:48.732043 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075974d-25d4-47a5-99ad-7bf4c39dfbf1 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:48.732586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075974d-25d4-47a5-99ad-7bf4c39dfbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:48.733266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075974d-25d4-47a5-99ad-7bf4c39dfbf1 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:48.736493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 675/1373] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:59:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 16:59:48.745603 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee496bc5-f78a-47f0-b61c-a4bb60d16170 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:48.746118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee496bc5-f78a-47f0-b61c-a4bb60d16170 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:48.746872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee496bc5-f78a-47f0-b61c-a4bb60d16170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:48.767037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:48.837758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ee496bc5-f78a-47f0-b61c-a4bb60d16170 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:48.839875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ee496bc5-f78a-47f0-b61c-a4bb60d16170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:48.844783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fc668-a9a4-4cb6-8104-3aac6e4945e0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:59:48.955998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a770911-5186-4807-afb6-4ae583307ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:59:48.956524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a770911-5186-4807-afb6-4ae583307ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:48.956799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a770911-5186-4807-afb6-4ae583307ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:48.957343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 676/1374] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 16:59:48.957447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a770911-5186-4807-afb6-4ae583307ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:48.959023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6a770911-5186-4807-afb6-4ae583307ad8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:49.085162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:49.100958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a770911-5186-4807-afb6-4ae583307ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:59:49.102423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a770911-5186-4807-afb6-4ae583307ad8 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:59:49.134390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 699/1375] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 16:59:49.274807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee496bc5-f78a-47f0-b61c-a4bb60d16170 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:59:49.276287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ee496bc5-f78a-47f0-b61c-a4bb60d16170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:49.341835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:49.458882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee496bc5-f78a-47f0-b61c-a4bb60d16170 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:59:49.615850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68c355-5d76-48d6-bfac-a637919b1d8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:59:49.616625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68c355-5d76-48d6-bfac-a637919b1d8f service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:49.617037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68c355-5d76-48d6-bfac-a637919b1d8f service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:49.617477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68c355-5d76-48d6-bfac-a637919b1d8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:49.619534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3f68c355-5d76-48d6-bfac-a637919b1d8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:49.695389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:49.769902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68c355-5d76-48d6-bfac-a637919b1d8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:59:49.771302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68c355-5d76-48d6-bfac-a637919b1d8f service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:59:49.796034 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 677/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 16:59:52.181189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe4659-61fc-4cd1-9fed-7f6b98c76b33 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:52.181743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe4659-61fc-4cd1-9fed-7f6b98c76b33 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:52.181848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe4659-61fc-4cd1-9fed-7f6b98c76b33 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:52.183543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 700/1377] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:52.189587 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d9636-6e78-4dec-a815-1c9987a64fac None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:52.190102 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d9636-6e78-4dec-a815-1c9987a64fac None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:52.190187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d9636-6e78-4dec-a815-1c9987a64fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:52.275369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3a4d9636-6e78-4dec-a815-1c9987a64fac None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:52.277148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3a4d9636-6e78-4dec-a815-1c9987a64fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:52.383363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 678/1378] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:52.678563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe4659-61fc-4cd1-9fed-7f6b98c76b33 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:59:52.680140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c0fe4659-61fc-4cd1-9fed-7f6b98c76b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:52.990427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe4659-61fc-4cd1-9fed-7f6b98c76b33 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:59:53.065812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:53.170528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b533454-2123-4b9c-84bf-9f8f12aef3f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:59:53.171988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b533454-2123-4b9c-84bf-9f8f12aef3f4 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:53.181905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b533454-2123-4b9c-84bf-9f8f12aef3f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:53.181905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b533454-2123-4b9c-84bf-9f8f12aef3f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:53.181905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8b533454-2123-4b9c-84bf-9f8f12aef3f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:53.346034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b533454-2123-4b9c-84bf-9f8f12aef3f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:59:53.347814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b533454-2123-4b9c-84bf-9f8f12aef3f4 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:59:53.368291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 701/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 16:59:55.890514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0dba9-115e-4f19-8190-1f040ba6cc68 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:55.891353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0dba9-115e-4f19-8190-1f040ba6cc68 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:55.891353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0dba9-115e-4f19-8190-1f040ba6cc68 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:55.892730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 679/1380] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:55.901047 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496188ac-628f-41b2-b05c-92dce4a4acff None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:55.902096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496188ac-628f-41b2-b05c-92dce4a4acff None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:55.902420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496188ac-628f-41b2-b05c-92dce4a4acff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:55.988087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-496188ac-628f-41b2-b05c-92dce4a4acff None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:55.989421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-496188ac-628f-41b2-b05c-92dce4a4acff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:56.009322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:56.143791 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 702/1381] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 16:59:56.661385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0dba9-115e-4f19-8190-1f040ba6cc68 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 16:59:56.663294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-35f0dba9-115e-4f19-8190-1f040ba6cc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:56.700336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:56.974160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0dba9-115e-4f19-8190-1f040ba6cc68 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 16:59:57.123062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 16:59:57.154485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a43167-1605-4ac1-9e1f-fefced63a12f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 16:59:57.155429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a43167-1605-4ac1-9e1f-fefced63a12f service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:57.155955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a43167-1605-4ac1-9e1f-fefced63a12f service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:57.156475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a43167-1605-4ac1-9e1f-fefced63a12f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:57.159046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-78a43167-1605-4ac1-9e1f-fefced63a12f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:57.283685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a43167-1605-4ac1-9e1f-fefced63a12f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 16:59:57.285564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a43167-1605-4ac1-9e1f-fefced63a12f service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 16:59:57.309946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 680/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 16:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 16:59:59.781686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2e812-eaf2-42d1-92d4-acb571eb15b6 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:59.782764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2e812-eaf2-42d1-92d4-acb571eb15b6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:59.782764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2e812-eaf2-42d1-92d4-acb571eb15b6 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:59.783656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 703/1383] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 16:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 16:59:59.791325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031429a-8b3c-4ae1-a659-21d2cd1419dc None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 16:59:59.792649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031429a-8b3c-4ae1-a659-21d2cd1419dc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 16:59:59.793058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031429a-8b3c-4ae1-a659-21d2cd1419dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 16:59:59.906777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d031429a-8b3c-4ae1-a659-21d2cd1419dc None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 16:59:59.908716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d031429a-8b3c-4ae1-a659-21d2cd1419dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 16:59:59.996421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:00.060166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 681/1384] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 16:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:00.383964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2e812-eaf2-42d1-92d4-acb571eb15b6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:00.385781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bcd2e812-eaf2-42d1-92d4-acb571eb15b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:00.584390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:00.588074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2e812-eaf2-42d1-92d4-acb571eb15b6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:00.738483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d45be-a8b9-4647-8a2d-32798804d105 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:00.739501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d45be-a8b9-4647-8a2d-32798804d105 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:00.739993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d45be-a8b9-4647-8a2d-32798804d105 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:00.740486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d45be-a8b9-4647-8a2d-32798804d105 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:00.743749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-252d45be-a8b9-4647-8a2d-32798804d105 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:00.871046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d45be-a8b9-4647-8a2d-32798804d105 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:00.873778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d45be-a8b9-4647-8a2d-32798804d105 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:00.896869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 704/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:03.415454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdec66f-672e-4a64-acc5-f304bc1f0031 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:03.416296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdec66f-672e-4a64-acc5-f304bc1f0031 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:03.416296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdec66f-672e-4a64-acc5-f304bc1f0031 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:03.417310 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 682/1386] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:03.427420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8de529-3051-4d07-8be6-8006e014d2d7 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:03.427988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8de529-3051-4d07-8be6-8006e014d2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:03.428429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8de529-3051-4d07-8be6-8006e014d2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:03.437422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:03.517982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ec8de529-3051-4d07-8be6-8006e014d2d7 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:03.519304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ec8de529-3051-4d07-8be6-8006e014d2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:03.642239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 705/1387] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:03.938460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdec66f-672e-4a64-acc5-f304bc1f0031 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:03.940063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8cdec66f-672e-4a64-acc5-f304bc1f0031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:04.154362 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdec66f-672e-4a64-acc5-f304bc1f0031 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:04.213197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:04.306930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81960ae9-4de1-4274-b163-d6fde43f1b63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:04.308161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81960ae9-4de1-4274-b163-d6fde43f1b63 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:04.308420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81960ae9-4de1-4274-b163-d6fde43f1b63 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:04.308686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81960ae9-4de1-4274-b163-d6fde43f1b63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:04.310569 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-81960ae9-4de1-4274-b163-d6fde43f1b63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:04.431401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81960ae9-4de1-4274-b163-d6fde43f1b63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:04.433792 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81960ae9-4de1-4274-b163-d6fde43f1b63 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:04.458992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 683/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:06.061781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce6f36-dcdd-46bf-9850-f41ab919d79b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:06.062351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce6f36-dcdd-46bf-9850-f41ab919d79b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:06.062651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce6f36-dcdd-46bf-9850-f41ab919d79b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:06.065344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 706/1389] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:00:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:06.073962 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2f4f0-d469-4353-a514-ea64594ed196 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:06.074885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2f4f0-d469-4353-a514-ea64594ed196 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:06.075103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2f4f0-d469-4353-a514-ea64594ed196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:06.149572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:06.160564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-cbb2f4f0-d469-4353-a514-ea64594ed196 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:06.161952 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cbb2f4f0-d469-4353-a514-ea64594ed196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:06.285253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 684/1390] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:06.816528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95025812-260e-4106-9775-fdc13be435d4 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:06.817061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95025812-260e-4106-9775-fdc13be435d4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:06.817579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95025812-260e-4106-9775-fdc13be435d4 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:06.819501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 707/1391] 10.222.0.16 () {54 vars in 962 bytes} [Thu Sep 22 17:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:06.826875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83a184-7980-4b32-bd4b-1541bedfdc54 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:06.827815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83a184-7980-4b32-bd4b-1541bedfdc54 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:06.828566 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83a184-7980-4b32-bd4b-1541bedfdc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:06.949451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ff83a184-7980-4b32-bd4b-1541bedfdc54 None None] MFA Rules not processed for user `6103e9a612c84a9ea7d21cc5128fa40b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:06.951308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ff83a184-7980-4b32-bd4b-1541bedfdc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:07.062003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:07.072758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 685/1392] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Sep 22 17:00:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:07.082296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95025812-260e-4106-9775-fdc13be435d4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:07.084162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-95025812-260e-4106-9775-fdc13be435d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:07.235554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:07.270860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95025812-260e-4106-9775-fdc13be435d4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:07.292552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d387d-4a4e-4dce-a6cd-6949486638ae None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:07.293404 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d387d-4a4e-4dce-a6cd-6949486638ae None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:07.293404 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d387d-4a4e-4dce-a6cd-6949486638ae None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:07.295051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 686/1393] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:00:07.302156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688e42b-7755-4671-9b56-d632784d1555 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:07.302600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688e42b-7755-4671-9b56-d632784d1555 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:07.303090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688e42b-7755-4671-9b56-d632784d1555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:07.375990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a57d20-b2c7-40f9-9a88-e24d0da38e82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6103e9a612c84a9ea7d21cc5128fa40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:07.376604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a57d20-b2c7-40f9-9a88-e24d0da38e82 service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:07.376884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a57d20-b2c7-40f9-9a88-e24d0da38e82 service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:07.377165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a57d20-b2c7-40f9-9a88-e24d0da38e82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:07.378880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b0a57d20-b2c7-40f9-9a88-e24d0da38e82 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:07.396525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e688e42b-7755-4671-9b56-d632784d1555 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:07.397967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e688e42b-7755-4671-9b56-d632784d1555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:07.550590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a57d20-b2c7-40f9-9a88-e24d0da38e82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:07.552224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a57d20-b2c7-40f9-9a88-e24d0da38e82 service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:07.554172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 687/1394] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:00:07.576084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 708/1395] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:00:07.856911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688e42b-7755-4671-9b56-d632784d1555 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:07.859399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e688e42b-7755-4671-9b56-d632784d1555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:07.993321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:08.075384 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688e42b-7755-4671-9b56-d632784d1555 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:08.255948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83469ce8-2763-480a-a845-45fce91299f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:08.256647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83469ce8-2763-480a-a845-45fce91299f4 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:08.256981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83469ce8-2763-480a-a845-45fce91299f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:08.257327 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83469ce8-2763-480a-a845-45fce91299f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:08.259218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-83469ce8-2763-480a-a845-45fce91299f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:08.347243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:08.371846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83469ce8-2763-480a-a845-45fce91299f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:08.373498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83469ce8-2763-480a-a845-45fce91299f4 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:08.395311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 688/1396] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:08.515785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e34613-5a49-4c78-b2bf-4c86a01ebb4e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:08.516686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e34613-5a49-4c78-b2bf-4c86a01ebb4e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:08.516686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e34613-5a49-4c78-b2bf-4c86a01ebb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:08.543802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:08.628488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-91e34613-5a49-4c78-b2bf-4c86a01ebb4e None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:08.630611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-91e34613-5a49-4c78-b2bf-4c86a01ebb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:08.759696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 709/1397] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:00:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:00:08.808475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd0b64-80c1-4c59-9094-6a05cb9ced6a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:08.808981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd0b64-80c1-4c59-9094-6a05cb9ced6a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:08.809388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd0b64-80c1-4c59-9094-6a05cb9ced6a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:08.811172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 689/1398] 10.222.0.16 () {54 vars in 943 bytes} [Thu Sep 22 17:00:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:08.820976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400a2a3-861e-4df4-b51c-a364d4771095 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:08.821473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400a2a3-861e-4df4-b51c-a364d4771095 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:08.821817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400a2a3-861e-4df4-b51c-a364d4771095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:08.935181 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8400a2a3-861e-4df4-b51c-a364d4771095 None None] MFA Rules not processed for user `b1a54beb68ba4610976524684bc26d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:08.936987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8400a2a3-861e-4df4-b51c-a364d4771095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:09.001353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:09.110233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 710/1399] 10.222.0.16 () {58 vars in 1040 bytes} [Thu Sep 22 17:00:08 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:09.126694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd0b64-80c1-4c59-9094-6a05cb9ced6a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:09.128404 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1bbd0b64-80c1-4c59-9094-6a05cb9ced6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:09.373205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd0b64-80c1-4c59-9094-6a05cb9ced6a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:09.476899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:09.573919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773c17d-6443-490f-ba96-cb5ba07983a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:09.574633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773c17d-6443-490f-ba96-cb5ba07983a4 service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:09.575023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773c17d-6443-490f-ba96-cb5ba07983a4 service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:09.575422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773c17d-6443-490f-ba96-cb5ba07983a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:09.577571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1773c17d-6443-490f-ba96-cb5ba07983a4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:09.710135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773c17d-6443-490f-ba96-cb5ba07983a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:09.711806 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773c17d-6443-490f-ba96-cb5ba07983a4 service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:09.721853 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 690/1400] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:00:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 17:00:10.791241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810b469-602e-4fad-ab23-aaf256fa6a8f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:10.792240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810b469-602e-4fad-ab23-aaf256fa6a8f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:10.792240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810b469-602e-4fad-ab23-aaf256fa6a8f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:10.793800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 711/1401] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:00:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:00:10.801366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7156840-6f56-4a33-8caa-50953da92726 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:10.801954 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7156840-6f56-4a33-8caa-50953da92726 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:10.802378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7156840-6f56-4a33-8caa-50953da92726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:10.918377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b7156840-6f56-4a33-8caa-50953da92726 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:10.920468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b7156840-6f56-4a33-8caa-50953da92726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:11.007912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:11.085582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 691/1402] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:11.418643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810b469-602e-4fad-ab23-aaf256fa6a8f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:11.421085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0810b469-602e-4fad-ab23-aaf256fa6a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:11.705585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810b469-602e-4fad-ab23-aaf256fa6a8f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:11.727965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:11.965858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c805905-3c11-4738-8329-4ccfa8e08a48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:11.966892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c805905-3c11-4738-8329-4ccfa8e08a48 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:11.966892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c805905-3c11-4738-8329-4ccfa8e08a48 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:11.966892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c805905-3c11-4738-8329-4ccfa8e08a48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:11.968655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4c805905-3c11-4738-8329-4ccfa8e08a48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:12.109714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c805905-3c11-4738-8329-4ccfa8e08a48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:12.112353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c805905-3c11-4738-8329-4ccfa8e08a48 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:12.142072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 712/1403] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:14.755515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc4812-93d8-48ff-92d9-d761549b3402 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:14.756389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc4812-93d8-48ff-92d9-d761549b3402 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:14.756389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc4812-93d8-48ff-92d9-d761549b3402 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:14.759243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 692/1404] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:00:14 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:14.765286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18526d8-4487-4e25-baf6-9bf1d920bc3f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:14.766147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18526d8-4487-4e25-baf6-9bf1d920bc3f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:14.766147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18526d8-4487-4e25-baf6-9bf1d920bc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:14.798817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:14.861571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d18526d8-4487-4e25-baf6-9bf1d920bc3f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:14.863229 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d18526d8-4487-4e25-baf6-9bf1d920bc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:15.079451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 713/1405] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:15.414160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc4812-93d8-48ff-92d9-d761549b3402 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:15.415927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a7fc4812-93d8-48ff-92d9-d761549b3402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:15.669414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc4812-93d8-48ff-92d9-d761549b3402 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:15.717562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:15.922257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8d774-72cd-4b7f-bff9-843005a302f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:15.923349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8d774-72cd-4b7f-bff9-843005a302f2 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:15.923790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8d774-72cd-4b7f-bff9-843005a302f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:15.924198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8d774-72cd-4b7f-bff9-843005a302f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:15.926118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-72c8d774-72cd-4b7f-bff9-843005a302f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:16.066693 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8d774-72cd-4b7f-bff9-843005a302f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:16.068594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8d774-72cd-4b7f-bff9-843005a302f2 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:16.099870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 693/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:19.948830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aa20f-79f5-4840-8780-82f9c6765186 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:19.949830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aa20f-79f5-4840-8780-82f9c6765186 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:19.950115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aa20f-79f5-4840-8780-82f9c6765186 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:19.955463 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 714/1407] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:00:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:19.958377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6c896-edd5-4008-9259-e6f5f2541ef7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:19.959615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6c896-edd5-4008-9259-e6f5f2541ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:19.960051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6c896-edd5-4008-9259-e6f5f2541ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:20.037102 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:20.084935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-95c6c896-edd5-4008-9259-e6f5f2541ef7 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:20.086301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-95c6c896-edd5-4008-9259-e6f5f2541ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:20.254991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 694/1408] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:20.574536 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aa20f-79f5-4840-8780-82f9c6765186 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:20.577559 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d41aa20f-79f5-4840-8780-82f9c6765186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:20.717156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:20.793385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aa20f-79f5-4840-8780-82f9c6765186 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:21.009246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1527e-d717-4a4e-a46d-0651fcdbb218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:21.010474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1527e-d717-4a4e-a46d-0651fcdbb218 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:21.010474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1527e-d717-4a4e-a46d-0651fcdbb218 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:21.010885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1527e-d717-4a4e-a46d-0651fcdbb218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:21.012881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1fd1527e-d717-4a4e-a46d-0651fcdbb218 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:21.097208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:21.131622 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1527e-d717-4a4e-a46d-0651fcdbb218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:21.133165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1527e-d717-4a4e-a46d-0651fcdbb218 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:21.158020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 715/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:00:30.251257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=52 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:31.156638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=18 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:35.446380 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c9044-bbfb-490a-8f85-32604dff05e3 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:35.447022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c9044-bbfb-490a-8f85-32604dff05e3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:35.447345 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c9044-bbfb-490a-8f85-32604dff05e3 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:35.449902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 695/1410] 10.222.0.16 () {54 vars in 918 bytes} [Thu Sep 22 17:00:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:35.456667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602e109-d61a-4b2e-9c41-c116c75d440c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:35.457052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602e109-d61a-4b2e-9c41-c116c75d440c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:35.457275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602e109-d61a-4b2e-9c41-c116c75d440c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:35.540094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c602e109-d61a-4b2e-9c41-c116c75d440c None None] MFA Rules not processed for user `81088a327fd448ad8fec555f3d914be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:35.541931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c602e109-d61a-4b2e-9c41-c116c75d440c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:35.665073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 716/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Sep 22 17:00:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:00:35.682382 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe7d3b-4447-4909-9733-5b13a0199d03 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:35.682836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe7d3b-4447-4909-9733-5b13a0199d03 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:35.683348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe7d3b-4447-4909-9733-5b13a0199d03 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:35.685112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 696/1412] 10.222.0.16 () {54 vars in 918 bytes} [Thu Sep 22 17:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:35.692741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686e3e0-ac3b-4e9b-b982-2f5540051317 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:35.693031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686e3e0-ac3b-4e9b-b982-2f5540051317 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:35.693307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686e3e0-ac3b-4e9b-b982-2f5540051317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:35.777950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:35.782049 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3686e3e0-ac3b-4e9b-b982-2f5540051317 None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:35.784018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3686e3e0-ac3b-4e9b-b982-2f5540051317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:35.915434 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 717/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Sep 22 17:00:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:00:38.677903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d6fab-bc1d-4da3-954f-c6969fe8d2a4 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:38.678822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d6fab-bc1d-4da3-954f-c6969fe8d2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:38.678822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d6fab-bc1d-4da3-954f-c6969fe8d2a4 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:38.680646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:00:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:38.687474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c29ac-9601-448f-8979-51ee23204101 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:38.688068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c29ac-9601-448f-8979-51ee23204101 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:38.688068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c29ac-9601-448f-8979-51ee23204101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:38.768808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-cf2c29ac-9601-448f-8979-51ee23204101 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:38.770056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cf2c29ac-9601-448f-8979-51ee23204101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:38.836949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:38.888234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:39.224515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d6fab-bc1d-4da3-954f-c6969fe8d2a4 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:39.226146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-421d6fab-bc1d-4da3-954f-c6969fe8d2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:39.417601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d6fab-bc1d-4da3-954f-c6969fe8d2a4 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:39.505963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:39.553961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625b0ef-70ca-41af-87cf-2c5e089728d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:39.554584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625b0ef-70ca-41af-87cf-2c5e089728d7 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:39.554862 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625b0ef-70ca-41af-87cf-2c5e089728d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:39.555170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625b0ef-70ca-41af-87cf-2c5e089728d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:39.556852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e625b0ef-70ca-41af-87cf-2c5e089728d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:39.665590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625b0ef-70ca-41af-87cf-2c5e089728d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:39.667893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625b0ef-70ca-41af-87cf-2c5e089728d7 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:39.690191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 698/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:40.931639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cddac-73a5-4bdf-8ce7-aff7f3a2b170 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:40.932318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cddac-73a5-4bdf-8ce7-aff7f3a2b170 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:40.932318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cddac-73a5-4bdf-8ce7-aff7f3a2b170 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:40.933634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:40.942023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b76dc-7d28-4f2f-b3fe-34cf66d8c3dc None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:40.942768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b76dc-7d28-4f2f-b3fe-34cf66d8c3dc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:40.942939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b76dc-7d28-4f2f-b3fe-34cf66d8c3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:41.068452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-050b76dc-7d28-4f2f-b3fe-34cf66d8c3dc None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:41.069719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-050b76dc-7d28-4f2f-b3fe-34cf66d8c3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:41.103845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:41.194747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:41.479121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cddac-73a5-4bdf-8ce7-aff7f3a2b170 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:41.481834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-586cddac-73a5-4bdf-8ce7-aff7f3a2b170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:41.672883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cddac-73a5-4bdf-8ce7-aff7f3a2b170 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:41.724293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:41.817371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b24bf2-e234-4b90-b61c-59f90e56def5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:41.817982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b24bf2-e234-4b90-b61c-59f90e56def5 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:41.818257 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b24bf2-e234-4b90-b61c-59f90e56def5 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:41.818522 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b24bf2-e234-4b90-b61c-59f90e56def5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:41.820339 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c5b24bf2-e234-4b90-b61c-59f90e56def5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:41.927324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b24bf2-e234-4b90-b61c-59f90e56def5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:41.928991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b24bf2-e234-4b90-b61c-59f90e56def5 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:41.950207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 720/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:43.226594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e16d0c-67af-41aa-b891-88a3bb532e38 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:43.227275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e16d0c-67af-41aa-b891-88a3bb532e38 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:43.227275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e16d0c-67af-41aa-b891-88a3bb532e38 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:43.228547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:00:43.236226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040e63a-2274-4625-be50-f734ebaaa1ad None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:43.236830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040e63a-2274-4625-be50-f734ebaaa1ad None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:43.236830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040e63a-2274-4625-be50-f734ebaaa1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:43.301510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:43.329092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f040e63a-2274-4625-be50-f734ebaaa1ad None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:43.330551 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f040e63a-2274-4625-be50-f734ebaaa1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:43.445810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:43.612203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e16d0c-67af-41aa-b891-88a3bb532e38 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:43.614453 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e8e16d0c-67af-41aa-b891-88a3bb532e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:43.799448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e16d0c-67af-41aa-b891-88a3bb532e38 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:43.814337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:43.971754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2330595-df80-4d46-b334-235847088312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:43.972801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2330595-df80-4d46-b334-235847088312 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:43.973295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2330595-df80-4d46-b334-235847088312 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:43.973790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2330595-df80-4d46-b334-235847088312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:43.977135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f2330595-df80-4d46-b334-235847088312 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:44.100358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2330595-df80-4d46-b334-235847088312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:44.101980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2330595-df80-4d46-b334-235847088312 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:44.123053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 701/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:00:45.296925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21073b-d38d-4b10-b7e5-cbba967a20cf None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:45.297622 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21073b-d38d-4b10-b7e5-cbba967a20cf None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:45.298117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21073b-d38d-4b10-b7e5-cbba967a20cf None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:45.300462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:00:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:45.307932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e41452-ee60-45ea-97f8-fde107c70251 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:45.308577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e41452-ee60-45ea-97f8-fde107c70251 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:45.308577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e41452-ee60-45ea-97f8-fde107c70251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:45.330836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:45.424580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-40e41452-ee60-45ea-97f8-fde107c70251 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:45.426780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-40e41452-ee60-45ea-97f8-fde107c70251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:45.574628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:45.726112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21073b-d38d-4b10-b7e5-cbba967a20cf None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:45.727667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7b21073b-d38d-4b10-b7e5-cbba967a20cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:45.852719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:45.897497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21073b-d38d-4b10-b7e5-cbba967a20cf None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:46.043010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db4f40-27b0-44f9-b630-218da5fdad5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:46.043630 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4f40-27b0-44f9-b630-218da5fdad5c service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:46.043912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4f40-27b0-44f9-b630-218da5fdad5c service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:46.044177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4f40-27b0-44f9-b630-218da5fdad5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:46.045864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-95db4f40-27b0-44f9-b630-218da5fdad5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:46.149938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:46.166315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db4f40-27b0-44f9-b630-218da5fdad5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:46.168124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db4f40-27b0-44f9-b630-218da5fdad5c service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:46.190298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 723/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:47.393459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6147f55-197a-4699-b7b4-74498079bbeb None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:47.394311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6147f55-197a-4699-b7b4-74498079bbeb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:47.394311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6147f55-197a-4699-b7b4-74498079bbeb None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:47.395923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:47.405941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252cd79-3874-4c3f-ac7f-c169ff3f661b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:47.406529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252cd79-3874-4c3f-ac7f-c169ff3f661b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:47.406529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252cd79-3874-4c3f-ac7f-c169ff3f661b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:47.497137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6252cd79-3874-4c3f-ac7f-c169ff3f661b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:47.498367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6252cd79-3874-4c3f-ac7f-c169ff3f661b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:47.628514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:47.792787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6147f55-197a-4699-b7b4-74498079bbeb None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:47.794354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d6147f55-197a-4699-b7b4-74498079bbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:47.893848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:47.967658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6147f55-197a-4699-b7b4-74498079bbeb None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:48.125969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b99a58-8542-4874-b0a9-977f830bd47f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:48.126906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b99a58-8542-4874-b0a9-977f830bd47f service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:48.127420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b99a58-8542-4874-b0a9-977f830bd47f service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:48.127867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b99a58-8542-4874-b0a9-977f830bd47f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:48.130883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d1b99a58-8542-4874-b0a9-977f830bd47f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:48.226510 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:48.278674 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b99a58-8542-4874-b0a9-977f830bd47f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:48.280504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b99a58-8542-4874-b0a9-977f830bd47f service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:48.301374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 704/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:49.431705 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1c0e-b20d-479e-b73e-f67fca6d6fa6 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:49.432555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1c0e-b20d-479e-b73e-f67fca6d6fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:49.432732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1c0e-b20d-479e-b73e-f67fca6d6fa6 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:49.434368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:00:49.441963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da9088-37b7-47f2-8d19-7d8877e210ce None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:49.442893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da9088-37b7-47f2-8d19-7d8877e210ce None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:49.442893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da9088-37b7-47f2-8d19-7d8877e210ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:49.559901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f2da9088-37b7-47f2-8d19-7d8877e210ce None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:49.561184 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f2da9088-37b7-47f2-8d19-7d8877e210ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:49.676019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:00:49.832130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe1c0e-b20d-479e-b73e-f67fca6d6fa6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:49.833691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-befe1c0e-b20d-479e-b73e-f67fca6d6fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:49.893641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:50.009014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe1c0e-b20d-479e-b73e-f67fca6d6fa6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:50.142677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55ceb0-df8e-4bfe-97e2-345c280b3c8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:50.143325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55ceb0-df8e-4bfe-97e2-345c280b3c8c service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:50.143599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55ceb0-df8e-4bfe-97e2-345c280b3c8c service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:50.143864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55ceb0-df8e-4bfe-97e2-345c280b3c8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:50.145592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3b55ceb0-df8e-4bfe-97e2-345c280b3c8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:50.153888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:50.253543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55ceb0-df8e-4bfe-97e2-345c280b3c8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:50.255294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55ceb0-df8e-4bfe-97e2-345c280b3c8c service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:50.276207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 726/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:52.190888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29734320-8c1f-4967-99b6-a63d85a9a326 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:52.191565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29734320-8c1f-4967-99b6-a63d85a9a326 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:52.191565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29734320-8c1f-4967-99b6-a63d85a9a326 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:52.192926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:00:52.201011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d033-cc05-403f-a494-83df4de269d1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:52.201843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d033-cc05-403f-a494-83df4de269d1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:52.201843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d033-cc05-403f-a494-83df4de269d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:52.289433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f944d033-cc05-403f-a494-83df4de269d1 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:52.290679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f944d033-cc05-403f-a494-83df4de269d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:52.370128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:52.403376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:52.563674 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29734320-8c1f-4967-99b6-a63d85a9a326 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:52.565236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-29734320-8c1f-4967-99b6-a63d85a9a326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:52.585654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:52.739126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29734320-8c1f-4967-99b6-a63d85a9a326 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:52.849021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:52.879348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d89937-be99-4522-bbfe-bbc9db7e989e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:52.879961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d89937-be99-4522-bbfe-bbc9db7e989e service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:52.880238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d89937-be99-4522-bbfe-bbc9db7e989e service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:52.880509 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d89937-be99-4522-bbfe-bbc9db7e989e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:52.882170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-17d89937-be99-4522-bbfe-bbc9db7e989e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:53.030681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d89937-be99-4522-bbfe-bbc9db7e989e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:53.032371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d89937-be99-4522-bbfe-bbc9db7e989e service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:53.054286 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 707/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:54.194723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbe781-a243-4a5b-b66b-33fba15b290e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:54.195790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbe781-a243-4a5b-b66b-33fba15b290e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:54.195790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbe781-a243-4a5b-b66b-33fba15b290e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:54.197845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:00:54.208356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb45b6-7163-4de0-a3f9-5e2a96c3bf41 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:54.209243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb45b6-7163-4de0-a3f9-5e2a96c3bf41 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:54.209243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb45b6-7163-4de0-a3f9-5e2a96c3bf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:54.296985 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b3cb45b6-7163-4de0-a3f9-5e2a96c3bf41 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:54.298671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b3cb45b6-7163-4de0-a3f9-5e2a96c3bf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:54.347964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:54.412692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:54.568843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbe781-a243-4a5b-b66b-33fba15b290e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:54.570286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-14cbe781-a243-4a5b-b66b-33fba15b290e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:54.757443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbe781-a243-4a5b-b66b-33fba15b290e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:54.822722 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:54.915667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ee23c-c925-4b3a-a251-50b3177615b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:54.916850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ee23c-c925-4b3a-a251-50b3177615b7 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:54.917159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ee23c-c925-4b3a-a251-50b3177615b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:54.917431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ee23c-c925-4b3a-a251-50b3177615b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:54.919158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a53ee23c-c925-4b3a-a251-50b3177615b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:55.042369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ee23c-c925-4b3a-a251-50b3177615b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:55.044037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ee23c-c925-4b3a-a251-50b3177615b7 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:55.066773 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 729/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:56.209726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c0e49-21e2-43f3-8d60-4c19e990ca7c None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:56.210314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c0e49-21e2-43f3-8d60-4c19e990ca7c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:56.210314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c0e49-21e2-43f3-8d60-4c19e990ca7c None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:56.211941 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:00:56.220593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efb0b1-58a1-4224-9a54-d87814a76ec6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:56.221198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efb0b1-58a1-4224-9a54-d87814a76ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:56.221338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efb0b1-58a1-4224-9a54-d87814a76ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:56.312187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-65efb0b1-58a1-4224-9a54-d87814a76ec6 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:56.314177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-65efb0b1-58a1-4224-9a54-d87814a76ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:56.316577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:56.462267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:00:56.615802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c0e49-21e2-43f3-8d60-4c19e990ca7c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:00:56.618438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-430c0e49-21e2-43f3-8d60-4c19e990ca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:56.846922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c0e49-21e2-43f3-8d60-4c19e990ca7c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:00:56.879593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:57.009069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01037fef-c54e-4974-a5ec-3364ce76db40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:00:57.010093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01037fef-c54e-4974-a5ec-3364ce76db40 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:57.010578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01037fef-c54e-4974-a5ec-3364ce76db40 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:57.011085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01037fef-c54e-4974-a5ec-3364ce76db40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:57.014409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-01037fef-c54e-4974-a5ec-3364ce76db40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:57.153084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01037fef-c54e-4974-a5ec-3364ce76db40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:00:57.154727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01037fef-c54e-4974-a5ec-3364ce76db40 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:00:57.176050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 710/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:00:59.555234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8753081-b677-4cdc-abf4-6c50a5228756 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:59.555933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8753081-b677-4cdc-abf4-6c50a5228756 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:59.555933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8753081-b677-4cdc-abf4-6c50a5228756 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:59.557250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:00:59.563776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dab94-369a-4b69-935f-8bd9c107fc4c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:00:59.564611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dab94-369a-4b69-935f-8bd9c107fc4c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:00:59.564611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dab94-369a-4b69-935f-8bd9c107fc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:00:59.615888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:00:59.652788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-784dab94-369a-4b69-935f-8bd9c107fc4c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:00:59.654863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-784dab94-369a-4b69-935f-8bd9c107fc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:00:59.794519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:01:00.089169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8753081-b677-4cdc-abf4-6c50a5228756 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:00.090751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a8753081-b677-4cdc-abf4-6c50a5228756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:00.219997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8753081-b677-4cdc-abf4-6c50a5228756 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:00.259127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:00.310631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbdacd-83a4-460d-b10d-6e6f163067ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:00.311381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbdacd-83a4-460d-b10d-6e6f163067ff service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:00.311708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbdacd-83a4-460d-b10d-6e6f163067ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:00.312020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbdacd-83a4-460d-b10d-6e6f163067ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:00.313764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-21fbdacd-83a4-460d-b10d-6e6f163067ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:00.597414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbdacd-83a4-460d-b10d-6e6f163067ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:00.598870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbdacd-83a4-460d-b10d-6e6f163067ff service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:00.622806 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 732/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:01:03.062334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5509f0-fa76-4aac-a18e-124be5d9dd2e None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:03.063024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5509f0-fa76-4aac-a18e-124be5d9dd2e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:03.063024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5509f0-fa76-4aac-a18e-124be5d9dd2e None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:03.064448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:01:03.071219 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501412fa-6160-41c1-9871-ee0a51748151 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:03.072664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501412fa-6160-41c1-9871-ee0a51748151 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:03.072664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501412fa-6160-41c1-9871-ee0a51748151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:03.151750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-501412fa-6160-41c1-9871-ee0a51748151 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:03.152989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-501412fa-6160-41c1-9871-ee0a51748151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:03.176668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:03.278330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:01:03.569190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5509f0-fa76-4aac-a18e-124be5d9dd2e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:03.570753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3c5509f0-fa76-4aac-a18e-124be5d9dd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:03.710374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5509f0-fa76-4aac-a18e-124be5d9dd2e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:03.725716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:03.820952 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e79c1-3e4e-4191-9cc7-39b242826d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:03.821926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e79c1-3e4e-4191-9cc7-39b242826d4b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:03.822417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e79c1-3e4e-4191-9cc7-39b242826d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:03.822899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e79c1-3e4e-4191-9cc7-39b242826d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:03.826133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2b7e79c1-3e4e-4191-9cc7-39b242826d4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:04.015221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e79c1-3e4e-4191-9cc7-39b242826d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:04.017936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e79c1-3e4e-4191-9cc7-39b242826d4b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:04.054367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 713/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:01:06.694349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa5a86-793a-4bf3-bfd6-79a557b01aba None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:06.695188 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa5a86-793a-4bf3-bfd6-79a557b01aba None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:06.695188 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa5a86-793a-4bf3-bfd6-79a557b01aba None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:06.696762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:01:06.703184 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b4426-b70d-483e-9504-105110800d88 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:06.704010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b4426-b70d-483e-9504-105110800d88 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:06.704010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b4426-b70d-483e-9504-105110800d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:06.779623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:06.801567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d47b4426-b70d-483e-9504-105110800d88 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:06.802808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d47b4426-b70d-483e-9504-105110800d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:06.916491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:01:07.207510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa5a86-793a-4bf3-bfd6-79a557b01aba None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:07.209186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b6aa5a86-793a-4bf3-bfd6-79a557b01aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:07.328921 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa5a86-793a-4bf3-bfd6-79a557b01aba None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:07.375959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:07.438140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed2ece-d948-4ea0-bcef-f0332006c5b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:07.439213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed2ece-d948-4ea0-bcef-f0332006c5b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:07.439663 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed2ece-d948-4ea0-bcef-f0332006c5b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:07.440112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed2ece-d948-4ea0-bcef-f0332006c5b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:07.443235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-58ed2ece-d948-4ea0-bcef-f0332006c5b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:07.599166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed2ece-d948-4ea0-bcef-f0332006c5b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:07.600694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed2ece-d948-4ea0-bcef-f0332006c5b7 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:07.622680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 735/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:01:10.072409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd3ddc-99ba-44a8-bfbb-72ca4807e3ef None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:10.073005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd3ddc-99ba-44a8-bfbb-72ca4807e3ef None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:10.073280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd3ddc-99ba-44a8-bfbb-72ca4807e3ef None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:10.075623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:01:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:01:10.082799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf922878-f00b-4039-8f31-b933a40ac48e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:10.083422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf922878-f00b-4039-8f31-b933a40ac48e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:10.083422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf922878-f00b-4039-8f31-b933a40ac48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:10.161171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bf922878-f00b-4039-8f31-b933a40ac48e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:10.162426 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bf922878-f00b-4039-8f31-b933a40ac48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:10.185893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:10.273941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:01:10.577190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd3ddc-99ba-44a8-bfbb-72ca4807e3ef None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:10.579226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1edd3ddc-99ba-44a8-bfbb-72ca4807e3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:10.701622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd3ddc-99ba-44a8-bfbb-72ca4807e3ef None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:10.727304 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:10.799535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d7f57-7d47-4a1f-bf0d-32c2bee5e283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:10.800156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d7f57-7d47-4a1f-bf0d-32c2bee5e283 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:10.800434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d7f57-7d47-4a1f-bf0d-32c2bee5e283 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:10.800702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d7f57-7d47-4a1f-bf0d-32c2bee5e283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:10.802714 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-168d7f57-7d47-4a1f-bf0d-32c2bee5e283 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:10.938862 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d7f57-7d47-4a1f-bf0d-32c2bee5e283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:10.940392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d7f57-7d47-4a1f-bf0d-32c2bee5e283 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:10.962729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 716/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:01:13.666561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cd4b4-48c9-4b87-9d0f-b98141d0cb9b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:13.667359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cd4b4-48c9-4b87-9d0f-b98141d0cb9b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:13.667601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cd4b4-48c9-4b87-9d0f-b98141d0cb9b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:13.670192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:01:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:01:13.676969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ec32-8dd7-455d-8952-fe4839b57608 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:13.677747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ec32-8dd7-455d-8952-fe4839b57608 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:13.677747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ec32-8dd7-455d-8952-fe4839b57608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:13.754830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:13.758489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c1a7ec32-8dd7-455d-8952-fe4839b57608 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:13.765542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c1a7ec32-8dd7-455d-8952-fe4839b57608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:13.877224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:01:14.172311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cd4b4-48c9-4b87-9d0f-b98141d0cb9b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:14.174250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cf2cd4b4-48c9-4b87-9d0f-b98141d0cb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:14.308103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cd4b4-48c9-4b87-9d0f-b98141d0cb9b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:14.366510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:14.397797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111b451-9db6-4b38-aeea-fd51d5a9f534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:14.398406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b451-9db6-4b38-aeea-fd51d5a9f534 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:14.398685 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b451-9db6-4b38-aeea-fd51d5a9f534 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:14.398989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b451-9db6-4b38-aeea-fd51d5a9f534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:14.401154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f111b451-9db6-4b38-aeea-fd51d5a9f534 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:14.539765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111b451-9db6-4b38-aeea-fd51d5a9f534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:14.541288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111b451-9db6-4b38-aeea-fd51d5a9f534 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:14.567050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 738/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:01:16.977326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f2b84-3641-409f-99c1-163f1850f781 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:16.978568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f2b84-3641-409f-99c1-163f1850f781 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:16.978783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f2b84-3641-409f-99c1-163f1850f781 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:16.981266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:01:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:01:16.987540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff586b-3419-4869-b794-56e588d71422 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:16.988406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff586b-3419-4869-b794-56e588d71422 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:16.988406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff586b-3419-4869-b794-56e588d71422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:17.100150 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-abff586b-3419-4869-b794-56e588d71422 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:17.102288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-abff586b-3419-4869-b794-56e588d71422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:17.158695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:17.240185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:01:17.537582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f2b84-3641-409f-99c1-163f1850f781 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:17.539266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e01f2b84-3641-409f-99c1-163f1850f781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:17.666410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f2b84-3641-409f-99c1-163f1850f781 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:17.699744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:17.757481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bd5cb-fcac-4bc1-a3d8-e939e10f67d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:17.758094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd5cb-fcac-4bc1-a3d8-e939e10f67d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:17.758364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd5cb-fcac-4bc1-a3d8-e939e10f67d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:17.758632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd5cb-fcac-4bc1-a3d8-e939e10f67d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:17.760837 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-082bd5cb-fcac-4bc1-a3d8-e939e10f67d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:17.892919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082bd5cb-fcac-4bc1-a3d8-e939e10f67d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:17.894293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082bd5cb-fcac-4bc1-a3d8-e939e10f67d9 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:17.916748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 719/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:01:20.217384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdaf8f-0cf6-4488-95b2-8f910cbd0889 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:20.218049 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdaf8f-0cf6-4488-95b2-8f910cbd0889 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:20.218049 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdaf8f-0cf6-4488-95b2-8f910cbd0889 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:20.219533 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:01:20.226190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235480b2-8cf6-4447-aad2-65b81df88019 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:20.226919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235480b2-8cf6-4447-aad2-65b81df88019 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:20.226919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235480b2-8cf6-4447-aad2-65b81df88019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:20.317089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-235480b2-8cf6-4447-aad2-65b81df88019 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:20.319266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-235480b2-8cf6-4447-aad2-65b81df88019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:20.336366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:20.437254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:01:20.729480 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdaf8f-0cf6-4488-95b2-8f910cbd0889 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:20.732388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d8bdaf8f-0cf6-4488-95b2-8f910cbd0889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:20.868224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdaf8f-0cf6-4488-95b2-8f910cbd0889 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:20.963532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:20.999364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba0647-2886-49b8-ae38-5ac3e30df1a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:21.000292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba0647-2886-49b8-ae38-5ac3e30df1a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:21.000749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba0647-2886-49b8-ae38-5ac3e30df1a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:21.001203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba0647-2886-49b8-ae38-5ac3e30df1a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:21.004310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-35ba0647-2886-49b8-ae38-5ac3e30df1a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:21.168652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba0647-2886-49b8-ae38-5ac3e30df1a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:21.171521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba0647-2886-49b8-ae38-5ac3e30df1a3 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:21.208946 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 741/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:01:23.670500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b5671-6cd9-4ec7-9feb-07d2f747afd5 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:23.671275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b5671-6cd9-4ec7-9feb-07d2f747afd5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:23.671275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b5671-6cd9-4ec7-9feb-07d2f747afd5 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:23.672753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:01:23.679706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44fb7e-3d86-4231-9704-0c7825ab3afd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:23.680636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44fb7e-3d86-4231-9704-0c7825ab3afd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:23.680636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44fb7e-3d86-4231-9704-0c7825ab3afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:23.760244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1a44fb7e-3d86-4231-9704-0c7825ab3afd None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:23.761740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1a44fb7e-3d86-4231-9704-0c7825ab3afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:23.813366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:23.881358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:01:24.170398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b5671-6cd9-4ec7-9feb-07d2f747afd5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:24.172726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2c5b5671-6cd9-4ec7-9feb-07d2f747afd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:24.304545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b5671-6cd9-4ec7-9feb-07d2f747afd5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:24.357363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:24.395547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a77936-092b-48b7-a582-a41d584358d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:24.396153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77936-092b-48b7-a582-a41d584358d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:24.396435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77936-092b-48b7-a582-a41d584358d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:24.396724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77936-092b-48b7-a582-a41d584358d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:24.398326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f0a77936-092b-48b7-a582-a41d584358d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:24.523384 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a77936-092b-48b7-a582-a41d584358d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:24.525436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a77936-092b-48b7-a582-a41d584358d6 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:24.547144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 722/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:01:26.891433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba253d-0282-49db-9a63-c402c3ed10a5 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:26.892062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba253d-0282-49db-9a63-c402c3ed10a5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:26.892419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba253d-0282-49db-9a63-c402c3ed10a5 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:26.895276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:01:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:01:26.902373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75aca8-d77e-4325-94ed-6597b8dbc2c0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:26.903008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75aca8-d77e-4325-94ed-6597b8dbc2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:26.903008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75aca8-d77e-4325-94ed-6597b8dbc2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:27.132112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8e75aca8-d77e-4325-94ed-6597b8dbc2c0 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:27.133541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8e75aca8-d77e-4325-94ed-6597b8dbc2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:27.167099 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:27.278832 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:01:27.597439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba253d-0282-49db-9a63-c402c3ed10a5 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:27.599276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-33ba253d-0282-49db-9a63-c402c3ed10a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:27.743013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba253d-0282-49db-9a63-c402c3ed10a5 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:27.830428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:27.840626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68355898-4c39-4b30-99ad-46e3f7132e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:27.841381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68355898-4c39-4b30-99ad-46e3f7132e24 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:27.841757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68355898-4c39-4b30-99ad-46e3f7132e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:27.842125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68355898-4c39-4b30-99ad-46e3f7132e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:27.843998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-68355898-4c39-4b30-99ad-46e3f7132e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:27.986045 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68355898-4c39-4b30-99ad-46e3f7132e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:27.988478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68355898-4c39-4b30-99ad-46e3f7132e24 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:28.011169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 744/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:01:30.360981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b62d4-9be4-4f2d-85e8-cccc7cbfaba8 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:30.361596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b62d4-9be4-4f2d-85e8-cccc7cbfaba8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:30.361596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b62d4-9be4-4f2d-85e8-cccc7cbfaba8 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:30.362930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:01:30.369307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cbf58-8538-4161-9229-d0b4b83c2949 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:30.369878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cbf58-8538-4161-9229-d0b4b83c2949 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:30.369878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cbf58-8538-4161-9229-d0b4b83c2949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:30.451897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c85cbf58-8538-4161-9229-d0b4b83c2949 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:30.453163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c85cbf58-8538-4161-9229-d0b4b83c2949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:30.519221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:30.608022 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:01:30.917695 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b62d4-9be4-4f2d-85e8-cccc7cbfaba8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:30.919519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-da1b62d4-9be4-4f2d-85e8-cccc7cbfaba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:31.041924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b62d4-9be4-4f2d-85e8-cccc7cbfaba8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:31.104262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:31.151249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2ed72-4c7d-4937-b3e4-f3cb6c819a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:31.152190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2ed72-4c7d-4937-b3e4-f3cb6c819a9d service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:31.152632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2ed72-4c7d-4937-b3e4-f3cb6c819a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:31.153076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2ed72-4c7d-4937-b3e4-f3cb6c819a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:31.155906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-63e2ed72-4c7d-4937-b3e4-f3cb6c819a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:31.335445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2ed72-4c7d-4937-b3e4-f3cb6c819a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:31.338346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2ed72-4c7d-4937-b3e4-f3cb6c819a9d service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:31.369119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 725/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:01:40.604734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=22 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:41.365500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=66 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:52.448026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0398-1de7-47cd-a6c7-0e1473f50b0d None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:52.448826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0398-1de7-47cd-a6c7-0e1473f50b0d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:52.448826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0398-1de7-47cd-a6c7-0e1473f50b0d None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:52.449911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 746/1471] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:01:52.455937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d657f-2ec3-426a-8c30-8113b99d05ed None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:52.456762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d657f-2ec3-426a-8c30-8113b99d05ed None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:52.456930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d657f-2ec3-426a-8c30-8113b99d05ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:52.537493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1a2d657f-2ec3-426a-8c30-8113b99d05ed None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:52.539172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1a2d657f-2ec3-426a-8c30-8113b99d05ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:52.687067 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 726/1472] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:01:53.002846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d5083-e907-43ac-a001-774950f30d6e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:53.003159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d5083-e907-43ac-a001-774950f30d6e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:53.003429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d5083-e907-43ac-a001-774950f30d6e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:53.004776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 747/1473] 10.222.0.16 () {54 vars in 962 bytes} [Thu Sep 22 17:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:01:53.011931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e27eb8-493c-438d-8656-193266a8a653 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:53.012461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e27eb8-493c-438d-8656-193266a8a653 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:53.012914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e27eb8-493c-438d-8656-193266a8a653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:53.103625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e4e27eb8-493c-438d-8656-193266a8a653 None None] MFA Rules not processed for user `6103e9a612c84a9ea7d21cc5128fa40b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:53.104895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e4e27eb8-493c-438d-8656-193266a8a653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:53.129880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:53.195154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 727/1474] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Sep 22 17:01:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:01:53.204375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d5083-e907-43ac-a001-774950f30d6e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:53.205921 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-821d5083-e907-43ac-a001-774950f30d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:53.342020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d5083-e907-43ac-a001-774950f30d6e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:53.428631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad792a0b-b8d7-4760-acf5-940deec9e62d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6103e9a612c84a9ea7d21cc5128fa40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:53.429237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad792a0b-b8d7-4760-acf5-940deec9e62d service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:53.429511 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad792a0b-b8d7-4760-acf5-940deec9e62d service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:53.429810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad792a0b-b8d7-4760-acf5-940deec9e62d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:53.432057 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ad792a0b-b8d7-4760-acf5-940deec9e62d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:53.495420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:53.600171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad792a0b-b8d7-4760-acf5-940deec9e62d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:53.602882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad792a0b-b8d7-4760-acf5-940deec9e62d service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:53.640204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 748/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:01:56.082458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e982a34-80f0-420e-8b53-b5c3475a3ef7 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:56.083053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e982a34-80f0-420e-8b53-b5c3475a3ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:56.083053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e982a34-80f0-420e-8b53-b5c3475a3ef7 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:56.084505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 728/1476] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:01:56.090898 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38ef13-af13-4319-82c9-a19599b5f2d0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:56.091763 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38ef13-af13-4319-82c9-a19599b5f2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:56.091763 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38ef13-af13-4319-82c9-a19599b5f2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:56.183420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2c38ef13-af13-4319-82c9-a19599b5f2d0 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:56.185524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2c38ef13-af13-4319-82c9-a19599b5f2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:56.301212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 749/1477] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:01:56.582406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e982a34-80f0-420e-8b53-b5c3475a3ef7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:56.584019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8e982a34-80f0-420e-8b53-b5c3475a3ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:56.729462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e982a34-80f0-420e-8b53-b5c3475a3ef7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:01:56.796841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:56.850613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8bc0e-2f5d-43d9-8371-918a8ed98bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:01:56.851163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8bc0e-2f5d-43d9-8371-918a8ed98bfa admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:56.851442 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8bc0e-2f5d-43d9-8371-918a8ed98bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:56.851714 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8bc0e-2f5d-43d9-8371-918a8ed98bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:56.852465 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8bc0e-2f5d-43d9-8371-918a8ed98bfa admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:01:56.854236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-50c8bc0e-2f5d-43d9-8371-918a8ed98bfa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:01:56.854236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 22 17:01:56.854236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: Sep 22 17:01:56.855419 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8bc0e-2f5d-43d9-8371-918a8ed98bfa admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:01:56.877041 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 729/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 22 17:01:56 2022] GET /identity/v3/projects => generated 2553 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:01:59.214122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931f462-a344-4d28-9249-cb271b85c2bb None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:59.214743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931f462-a344-4d28-9249-cb271b85c2bb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:59.214743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931f462-a344-4d28-9249-cb271b85c2bb None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:59.216596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 750/1479] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:01:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:01:59.226531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857b665-748e-4379-afe0-d3efbe79b481 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:01:59.227163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857b665-748e-4379-afe0-d3efbe79b481 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:01:59.227568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857b665-748e-4379-afe0-d3efbe79b481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:01:59.355011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c857b665-748e-4379-afe0-d3efbe79b481 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:01:59.356374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c857b665-748e-4379-afe0-d3efbe79b481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:59.478301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 730/1480] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:01:59.792491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931f462-a344-4d28-9249-cb271b85c2bb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:01:59.794070 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0931f462-a344-4d28-9249-cb271b85c2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:01:59.805381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:01:59.973848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931f462-a344-4d28-9249-cb271b85c2bb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:02:00.066082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:02:00.101824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a17be-671f-47c9-bf59-3dc2791a3691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:02:00.102429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a17be-671f-47c9-bf59-3dc2791a3691 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:00.102707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a17be-671f-47c9-bf59-3dc2791a3691 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:00.103000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a17be-671f-47c9-bf59-3dc2791a3691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:00.104693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-645a17be-671f-47c9-bf59-3dc2791a3691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:00.209643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a17be-671f-47c9-bf59-3dc2791a3691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:02:00.211429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a17be-671f-47c9-bf59-3dc2791a3691 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:02:00.232030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 751/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:02:02.901690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d09328-0e77-44e3-9607-50c4048ea157 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:02.902855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d09328-0e77-44e3-9607-50c4048ea157 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:02.903236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d09328-0e77-44e3-9607-50c4048ea157 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:02.906029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 731/1482] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:02:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:02:02.912716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140dfa1-579f-40c4-8755-f45405494dcd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:02.913405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140dfa1-579f-40c4-8755-f45405494dcd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:02.913557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140dfa1-579f-40c4-8755-f45405494dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:02.994149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8140dfa1-579f-40c4-8755-f45405494dcd None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:02:02.995561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8140dfa1-579f-40c4-8755-f45405494dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:03.051542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:02:03.113063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 752/1483] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:02:03.410866 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d09328-0e77-44e3-9607-50c4048ea157 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:02:03.412593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a3d09328-0e77-44e3-9607-50c4048ea157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:03.441013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:02:03.546369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d09328-0e77-44e3-9607-50c4048ea157 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:02:03.637950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87ae3c-e306-4591-84e5-723dfbcc77a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:02:03.638566 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ae3c-e306-4591-84e5-723dfbcc77a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:03.638846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ae3c-e306-4591-84e5-723dfbcc77a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:03.639149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ae3c-e306-4591-84e5-723dfbcc77a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:03.640821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bb87ae3c-e306-4591-84e5-723dfbcc77a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:03.725806 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:02:03.767882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87ae3c-e306-4591-84e5-723dfbcc77a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:02:03.769444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87ae3c-e306-4591-84e5-723dfbcc77a6 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:02:03.790144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 732/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:02:06.238771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48bdcd-6b36-478d-a672-dbb995a3fb5c None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:06.239578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48bdcd-6b36-478d-a672-dbb995a3fb5c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:06.239881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48bdcd-6b36-478d-a672-dbb995a3fb5c None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:06.241493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 753/1485] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:02:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:02:06.249749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148c73b-416c-44e1-ba30-4db7afe548c2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:06.250339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148c73b-416c-44e1-ba30-4db7afe548c2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:06.250464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148c73b-416c-44e1-ba30-4db7afe548c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:06.339399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8148c73b-416c-44e1-ba30-4db7afe548c2 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:02:06.341578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8148c73b-416c-44e1-ba30-4db7afe548c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:06.487713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 733/1486] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:02:06.791659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48bdcd-6b36-478d-a672-dbb995a3fb5c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:02:06.793280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1f48bdcd-6b36-478d-a672-dbb995a3fb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:06.911053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48bdcd-6b36-478d-a672-dbb995a3fb5c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:02:07.024360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:02:07.026001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54380be-5b83-4d41-9d8b-fefe28d02de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:02:07.026624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54380be-5b83-4d41-9d8b-fefe28d02de1 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:07.026905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54380be-5b83-4d41-9d8b-fefe28d02de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:07.027216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54380be-5b83-4d41-9d8b-fefe28d02de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:07.029818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e54380be-5b83-4d41-9d8b-fefe28d02de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:07.175802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54380be-5b83-4d41-9d8b-fefe28d02de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:02:07.177261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54380be-5b83-4d41-9d8b-fefe28d02de1 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:02:07.199796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 754/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:02:09.672259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2da3a7-344b-47c0-b3e2-d76d06fb00ba None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:09.673198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2da3a7-344b-47c0-b3e2-d76d06fb00ba None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:09.673198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2da3a7-344b-47c0-b3e2-d76d06fb00ba None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:09.674503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 734/1488] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:02:09.681844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968ce2e-8a98-4a2a-91ce-64e89567a23b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:09.682479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968ce2e-8a98-4a2a-91ce-64e89567a23b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:09.682827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968ce2e-8a98-4a2a-91ce-64e89567a23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:09.786474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0968ce2e-8a98-4a2a-91ce-64e89567a23b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:02:09.787982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0968ce2e-8a98-4a2a-91ce-64e89567a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:09.852246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:02:09.905700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 755/1489] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:02:10.200573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2da3a7-344b-47c0-b3e2-d76d06fb00ba None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:02:10.202263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7b2da3a7-344b-47c0-b3e2-d76d06fb00ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:10.242685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:02:10.318452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2da3a7-344b-47c0-b3e2-d76d06fb00ba None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:02:10.397177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ae9cb-84fd-4880-aae5-ef6e501eec6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:02:10.397786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ae9cb-84fd-4880-aae5-ef6e501eec6e service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:10.398064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ae9cb-84fd-4880-aae5-ef6e501eec6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:10.398331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ae9cb-84fd-4880-aae5-ef6e501eec6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:10.400640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-414ae9cb-84fd-4880-aae5-ef6e501eec6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:10.492618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:02:10.526794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ae9cb-84fd-4880-aae5-ef6e501eec6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:02:10.528293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ae9cb-84fd-4880-aae5-ef6e501eec6e service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:02:10.547895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 735/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:02:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:02:13.104797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f0c56-b9d8-4afe-9623-0b108d1fddbf None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:13.105384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f0c56-b9d8-4afe-9623-0b108d1fddbf None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:13.105531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f0c56-b9d8-4afe-9623-0b108d1fddbf None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:13.107283 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 756/1491] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 22 17:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:02:13.113663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e20fe7-c71c-4983-9652-d4296cabaef7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:13.114191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e20fe7-c71c-4983-9652-d4296cabaef7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:13.114191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e20fe7-c71c-4983-9652-d4296cabaef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:13.187752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-64e20fe7-c71c-4983-9652-d4296cabaef7 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:02:13.188999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-64e20fe7-c71c-4983-9652-d4296cabaef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:13.298068 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 736/1492] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 22 17:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:02:13.608367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f0c56-b9d8-4afe-9623-0b108d1fddbf None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:02:13.609956 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a99f0c56-b9d8-4afe-9623-0b108d1fddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:13.771938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f0c56-b9d8-4afe-9623-0b108d1fddbf None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:02:13.872126 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e9a86-d620-4470-8a56-ebbd9747cc63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:02:13.872647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e9a86-d620-4470-8a56-ebbd9747cc63 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:02:13.872925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e9a86-d620-4470-8a56-ebbd9747cc63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:02:13.873192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e9a86-d620-4470-8a56-ebbd9747cc63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:02:13.874920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-735e9a86-d620-4470-8a56-ebbd9747cc63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:02:13.884853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:02:14.007359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e9a86-d620-4470-8a56-ebbd9747cc63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:02:14.010056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e9a86-d620-4470-8a56-ebbd9747cc63 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:02:14.033948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 757/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:02:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:02:23.296483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=82 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:02:24.031530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=50 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:03:32.871913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886272-dbca-4380-8a69-433c90b3563e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:32.872625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886272-dbca-4380-8a69-433c90b3563e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:32.872625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886272-dbca-4380-8a69-433c90b3563e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:32.966917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c0886272-dbca-4380-8a69-433c90b3563e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:03:32.968191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c0886272-dbca-4380-8a69-433c90b3563e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:03:33.090921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 737/1494] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:03:33.099091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c70d2-e60a-45ee-a767-267604b02ddb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:33.100240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c70d2-e60a-45ee-a767-267604b02ddb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:33.100240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c70d2-e60a-45ee-a767-267604b02ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:33.121167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3a0c70d2-e60a-45ee-a767-267604b02ddb None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:03:33.122394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3a0c70d2-e60a-45ee-a767-267604b02ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:03:33.131198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 758/1495] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:03:33.136069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0886272-dbca-4380-8a69-433c90b3563e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:03:33.146118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0886272-dbca-4380-8a69-433c90b3563e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:03:33.151990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b59e9-325a-40ea-85c7-d73781a162e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:03:33.152640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b59e9-325a-40ea-85c7-d73781a162e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:33.152964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b59e9-325a-40ea-85c7-d73781a162e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:33.153244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b59e9-325a-40ea-85c7-d73781a162e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:33.154015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b59e9-325a-40ea-85c7-d73781a162e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:03:33.154326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b59e9-325a-40ea-85c7-d73781a162e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:03:33.155446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-e78b59e9-325a-40ea-85c7-d73781a162e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:03:33.155446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 22 17:03:33.155446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: Sep 22 17:03:33.156366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b59e9-325a-40ea-85c7-d73781a162e9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:03:33.171068 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 738/1496] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:03:33.183809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c70d2-e60a-45ee-a767-267604b02ddb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:03:33.199203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c70d2-e60a-45ee-a767-267604b02ddb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:03:33.205719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ce773-3c65-4c3a-a7f1-14552cb5ca26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:03:33.206679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ce773-3c65-4c3a-a7f1-14552cb5ca26 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:33.207093 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ce773-3c65-4c3a-a7f1-14552cb5ca26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:33.207418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ce773-3c65-4c3a-a7f1-14552cb5ca26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:33.208294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ce773-3c65-4c3a-a7f1-14552cb5ca26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:03:33.211370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-c41ce773-3c65-4c3a-a7f1-14552cb5ca26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:03:33.211370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Sep 22 17:03:33.211370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: Sep 22 17:03:33.213500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ce773-3c65-4c3a-a7f1-14552cb5ca26 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:03:33.277707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 759/1497] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:03:33 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:03:33.282474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b59e9-325a-40ea-85c7-d73781a162e9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:03:33.294550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b59e9-325a-40ea-85c7-d73781a162e9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:03:33.351462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:03:33.358887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f412f-8536-4d66-ba8c-7624e378d584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:03:33.359611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f412f-8536-4d66-ba8c-7624e378d584 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:33.359891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f412f-8536-4d66-ba8c-7624e378d584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:33.360167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f412f-8536-4d66-ba8c-7624e378d584 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:33.360826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f412f-8536-4d66-ba8c-7624e378d584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:03:33.361895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-3e0f412f-8536-4d66-ba8c-7624e378d584 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:03:33.361895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 22 17:03:33.361895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: Sep 22 17:03:33.363147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f412f-8536-4d66-ba8c-7624e378d584 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:03:33.386595 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 739/1498] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:03:33 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:03:33.392847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ce773-3c65-4c3a-a7f1-14552cb5ca26 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:03:33.405082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ce773-3c65-4c3a-a7f1-14552cb5ca26 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:03:33.411993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad6fbe-2957-454a-8f5e-f867def105c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:03:33.412777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad6fbe-2957-454a-8f5e-f867def105c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:33.413054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad6fbe-2957-454a-8f5e-f867def105c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:33.413352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad6fbe-2957-454a-8f5e-f867def105c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:33.413998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad6fbe-2957-454a-8f5e-f867def105c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:03:33.415048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-a7ad6fbe-2957-454a-8f5e-f867def105c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:03:33.415048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Sep 22 17:03:33.415048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: Sep 22 17:03:33.415930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad6fbe-2957-454a-8f5e-f867def105c6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:03:33.430371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 760/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:03:33.435015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f412f-8536-4d66-ba8c-7624e378d584 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:03:33.449857 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f412f-8536-4d66-ba8c-7624e378d584 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:03:33.455287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521e0a3-87a4-461b-80bd-7b3681bf19ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:03:33.455967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e0a3-87a4-461b-80bd-7b3681bf19ba admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:33.456288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e0a3-87a4-461b-80bd-7b3681bf19ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:33.456600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521e0a3-87a4-461b-80bd-7b3681bf19ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:33.457293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521e0a3-87a4-461b-80bd-7b3681bf19ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:03:33.458356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-7521e0a3-87a4-461b-80bd-7b3681bf19ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:03:33.458356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 22 17:03:33.458356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: Sep 22 17:03:33.459427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521e0a3-87a4-461b-80bd-7b3681bf19ba admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:03:33.476753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 740/1500] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:03:33.481340 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad6fbe-2957-454a-8f5e-f867def105c6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:03:33.492720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad6fbe-2957-454a-8f5e-f867def105c6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:03:33.498575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724e40a-b002-4e79-8a2b-7810198b16db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:03:33.499687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724e40a-b002-4e79-8a2b-7810198b16db admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:33.500174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724e40a-b002-4e79-8a2b-7810198b16db admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:33.500662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724e40a-b002-4e79-8a2b-7810198b16db admin admin] PATH_INFO: `/v3/projects/e9efdb0139e1446cbfd73f470924932a/users/a1ccff22bc67493faf59dd69b362bf03/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:33.530000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724e40a-b002-4e79-8a2b-7810198b16db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9efdb0139e1446cbfd73f470924932a, user_id=a1ccff22bc67493faf59dd69b362bf03, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:03:33.532280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-5724e40a-b002-4e79-8a2b-7810198b16db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:03:33.532280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Sep 22 17:03:33.532280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: Sep 22 17:03:33.535730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724e40a-b002-4e79-8a2b-7810198b16db admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:03:33.553821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 761/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:03:33 2022] PUT /identity/v3/projects/e9efdb0139e1446cbfd73f470924932a/users/a1ccff22bc67493faf59dd69b362bf03/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:03:33.567805 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365a694-ee00-441c-b630-73dbae2ad3dd None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:33.568110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365a694-ee00-441c-b630-73dbae2ad3dd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:33.568353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365a694-ee00-441c-b630-73dbae2ad3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:33.598340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f365a694-ee00-441c-b630-73dbae2ad3dd None None] MFA Rules not processed for user `a1ccff22bc67493faf59dd69b362bf03`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:03:33.600094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f365a694-ee00-441c-b630-73dbae2ad3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:03:33.684955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 741/1502] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:03:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:03:33.699323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b822e1b-eeb0-4926-9be6-69226da216d8 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:33.699718 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b822e1b-eeb0-4926-9be6-69226da216d8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:33.699894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b822e1b-eeb0-4926-9be6-69226da216d8 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:33.701324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 762/1503] 10.222.0.16 () {54 vars in 963 bytes} [Thu Sep 22 17:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:03:33.707643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba6de9-71bc-4116-80db-9382da3b54a9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:33.708352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba6de9-71bc-4116-80db-9382da3b54a9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:33.708868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba6de9-71bc-4116-80db-9382da3b54a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:33.805503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-24ba6de9-71bc-4116-80db-9382da3b54a9 None None] MFA Rules not processed for user `ae1b8a51c9fa4f2594bdd79b58962b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:03:33.806874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-24ba6de9-71bc-4116-80db-9382da3b54a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:03:33.895198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 742/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Thu Sep 22 17:03:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:03:33.905912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b822e1b-eeb0-4926-9be6-69226da216d8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:03:33.917976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b822e1b-eeb0-4926-9be6-69226da216d8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:03:33.921705 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42006f1f-2758-4edb-bff1-f33de7da4ce6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:03:33.922315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42006f1f-2758-4edb-bff1-f33de7da4ce6 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:33.922590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42006f1f-2758-4edb-bff1-f33de7da4ce6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:33.922859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42006f1f-2758-4edb-bff1-f33de7da4ce6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:33.934686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42006f1f-2758-4edb-bff1-f33de7da4ce6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:03:33.936523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42006f1f-2758-4edb-bff1-f33de7da4ce6 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:03:33.947269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 763/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:03:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:03:34.000036 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d286d-9c2b-42dd-9577-3a4a360208df None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:34.000322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d286d-9c2b-42dd-9577-3a4a360208df None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:34.000597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d286d-9c2b-42dd-9577-3a4a360208df None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:34.001846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 743/1506] 10.222.0.16 () {52 vars in 830 bytes} [Thu Sep 22 17:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:03:34.005801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42006f1f-2758-4edb-bff1-f33de7da4ce6 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:03:34.073105 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42006f1f-2758-4edb-bff1-f33de7da4ce6 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:03:34.078295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:03:34.078962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:34.079255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:34.079526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] PATH_INFO: `/v3/users/a1ccff22bc67493faf59dd69b362bf03` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:34.080682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ccff22bc67493faf59dd69b362bf03)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:03:34.081712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:03:34.081712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Sep 22 17:03:34.081712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: Sep 22 17:03:34.082935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:03:34.160427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ccff22bc67493faf59dd69b362bf03'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:03:34.186859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ccff22bc67493faf59dd69b362bf03'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:03:34.202811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ccff22bc67493faf59dd69b362bf03'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:03:34.207463 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:03:34.216867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ccff22bc67493faf59dd69b362bf03'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:03:34.218386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ccff22bc67493faf59dd69b362bf03'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:03:34.226031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 764/1507] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:03:34 2022] DELETE /identity/v3/users/a1ccff22bc67493faf59dd69b362bf03 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:03:34.231836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c07bb-72a8-4562-b795-a3da06317b4a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:34.232422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c07bb-72a8-4562-b795-a3da06317b4a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:34.232936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c07bb-72a8-4562-b795-a3da06317b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:34.254969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-736c07bb-72a8-4562-b795-a3da06317b4a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:03:34.256773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-736c07bb-72a8-4562-b795-a3da06317b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:03:34.267277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:03:34.326311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 744/1508] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:03:34.337345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:03:34.338920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:03:34.435128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a0972-1bc7-417c-8184-20d4edbc10e0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:03:34.439441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d19d2b-321d-4262-9b7e-4cf73a349e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:03:34.440114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d19d2b-321d-4262-9b7e-4cf73a349e1a service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:34.440428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d19d2b-321d-4262-9b7e-4cf73a349e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:34.440785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d19d2b-321d-4262-9b7e-4cf73a349e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:34.451925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d19d2b-321d-4262-9b7e-4cf73a349e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:03:34.453534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d19d2b-321d-4262-9b7e-4cf73a349e1a service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:03:34.464886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 765/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:03:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:03:34.879833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c07bb-72a8-4562-b795-a3da06317b4a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:03:34.882236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-736c07bb-72a8-4562-b795-a3da06317b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:03:34.896234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c07bb-72a8-4562-b795-a3da06317b4a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:03:34.901969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:03:34.902628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:03:34.902901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:03:34.903212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] PATH_INFO: `/v3/projects/e9efdb0139e1446cbfd73f470924932a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:03:34.904485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9efdb0139e1446cbfd73f470924932a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:03:34.906078 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:03:34.906078 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 22 17:03:34.906078 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: Sep 22 17:03:34.907733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:03:34.998929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9efdb0139e1446cbfd73f470924932a', 'request_id': 'req-44046d62-4462-4167-8a98-b82f79e35522'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:03:35.008092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9efdb0139e1446cbfd73f470924932a', 'request_id': 'req-44046d62-4462-4167-8a98-b82f79e35522'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:03:35.034539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9efdb0139e1446cbfd73f470924932a', 'request_id': 'req-44046d62-4462-4167-8a98-b82f79e35522'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:03:35.051677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] The token cache is being invalidate because project e9efdb0139e1446cbfd73f470924932a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:03:35.051985 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:03:35.052686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-44046d62-4462-4167-8a98-b82f79e35522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:03:35.054358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 745/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:03:34 2022] DELETE /identity/v3/projects/e9efdb0139e1446cbfd73f470924932a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:03:44.460117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:03:45.048134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:05:01.896178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d379af4-c452-48c9-b427-0740d762d5a6 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:01.896950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d379af4-c452-48c9-b427-0740d762d5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:01.896950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d379af4-c452-48c9-b427-0740d762d5a6 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:01.898140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:05:01.904213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24f61c-44c7-4100-a0ed-942bbcef4e7d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:01.905004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24f61c-44c7-4100-a0ed-942bbcef4e7d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:01.905004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24f61c-44c7-4100-a0ed-942bbcef4e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:01.929568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3c24f61c-44c7-4100-a0ed-942bbcef4e7d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:05:01.930899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3c24f61c-44c7-4100-a0ed-942bbcef4e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:02.012309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:05:02.175938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d379af4-c452-48c9-b427-0740d762d5a6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:05:02.178009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5d379af4-c452-48c9-b427-0740d762d5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:02.305357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d379af4-c452-48c9-b427-0740d762d5a6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:05:02.311897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464eefd-cf68-48ce-ab7f-63bcd8bd0d6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:05:02.312534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464eefd-cf68-48ce-ab7f-63bcd8bd0d6d service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:02.312814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464eefd-cf68-48ce-ab7f-63bcd8bd0d6d service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:02.313089 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464eefd-cf68-48ce-ab7f-63bcd8bd0d6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:02.323454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464eefd-cf68-48ce-ab7f-63bcd8bd0d6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:05:02.325154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464eefd-cf68-48ce-ab7f-63bcd8bd0d6d service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:05:02.337201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 767/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:05:03.182180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2adce8-81bf-436b-b329-a796ddaa0a2a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:03.182821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2adce8-81bf-436b-b329-a796ddaa0a2a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:03.183025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2adce8-81bf-436b-b329-a796ddaa0a2a None None] PATH_INFO: `/v3` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:03.184417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 747/1514] 10.222.0.33 () {54 vars in 925 bytes} [Thu Sep 22 17:05:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 17:05:03.191324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9964a3-f4f0-4c44-9441-e9a06cf23c53 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:03.191923 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9964a3-f4f0-4c44-9441-e9a06cf23c53 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:03.192062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9964a3-f4f0-4c44-9441-e9a06cf23c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:03.255846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7d9964a3-f4f0-4c44-9441-e9a06cf23c53 None None] MFA Rules not processed for user `81088a327fd448ad8fec555f3d914be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:05:03.257103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7d9964a3-f4f0-4c44-9441-e9a06cf23c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:03.349702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 768/1515] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Sep 22 17:05:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:05:03.560024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe371-07b9-402f-9c4d-b9a459bbef45 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:03.560363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe371-07b9-402f-9c4d-b9a459bbef45 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:03.560744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe371-07b9-402f-9c4d-b9a459bbef45 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:03.562529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 748/1516] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:05:03.570988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d7d1c-dc41-440c-8106-f1aaf44afe17 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:03.571319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d7d1c-dc41-440c-8106-f1aaf44afe17 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:03.571641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d7d1c-dc41-440c-8106-f1aaf44afe17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:03.592361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3b7d7d1c-dc41-440c-8106-f1aaf44afe17 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:05:03.593626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3b7d7d1c-dc41-440c-8106-f1aaf44afe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:03.602638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 769/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:05:03.758277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe371-07b9-402f-9c4d-b9a459bbef45 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:05:03.776272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe371-07b9-402f-9c4d-b9a459bbef45 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:05:03.782403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d25146-f9fc-457a-aef9-05b6a7cc6ad1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:05:03.783136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d25146-f9fc-457a-aef9-05b6a7cc6ad1 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:03.783460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d25146-f9fc-457a-aef9-05b6a7cc6ad1 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:03.783827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d25146-f9fc-457a-aef9-05b6a7cc6ad1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:03.793343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d25146-f9fc-457a-aef9-05b6a7cc6ad1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:05:03.794983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d25146-f9fc-457a-aef9-05b6a7cc6ad1 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:05:03.809552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 749/1518] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:05:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:05:12.083379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddef36-6795-4862-9efa-ff0e635e2710 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:12.084164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddef36-6795-4862-9efa-ff0e635e2710 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:12.084164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddef36-6795-4862-9efa-ff0e635e2710 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:12.085348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 770/1519] 10.222.0.16 () {54 vars in 961 bytes} [Thu Sep 22 17:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:05:12.090929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c997f5f-bc9a-4a72-b57d-301d167e781f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:12.091572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c997f5f-bc9a-4a72-b57d-301d167e781f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:12.091702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c997f5f-bc9a-4a72-b57d-301d167e781f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:12.113406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6c997f5f-bc9a-4a72-b57d-301d167e781f None None] MFA Rules not processed for user `81088a327fd448ad8fec555f3d914be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:05:12.114716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6c997f5f-bc9a-4a72-b57d-301d167e781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:12.124193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 750/1520] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Sep 22 17:05:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:05:12.135342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddef36-6795-4862-9efa-ff0e635e2710 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:05:12.151341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddef36-6795-4862-9efa-ff0e635e2710 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:05:12.159405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e491495-955b-48f2-ad9b-f4fa50a68496 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:05:12.160515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e491495-955b-48f2-ad9b-f4fa50a68496 service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:12.161001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e491495-955b-48f2-ad9b-f4fa50a68496 service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:12.161517 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e491495-955b-48f2-ad9b-f4fa50a68496 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:12.178023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e491495-955b-48f2-ad9b-f4fa50a68496 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:05:12.181176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e491495-955b-48f2-ad9b-f4fa50a68496 service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:05:12.197192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 771/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:05:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:05:15.858870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8592b-290d-4ffe-9cca-9b7ca0f5c329 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:15.859571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8592b-290d-4ffe-9cca-9b7ca0f5c329 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:15.859571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8592b-290d-4ffe-9cca-9b7ca0f5c329 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:15.860844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:05:15.868562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479ea43-575b-41cd-ae08-a0dadce13180 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:15.869162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479ea43-575b-41cd-ae08-a0dadce13180 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:15.869162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479ea43-575b-41cd-ae08-a0dadce13180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:15.877131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:05:15.891769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0479ea43-575b-41cd-ae08-a0dadce13180 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:05:15.893013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0479ea43-575b-41cd-ae08-a0dadce13180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:15.901437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:05:16.059183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8592b-290d-4ffe-9cca-9b7ca0f5c329 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:05:16.085722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8592b-290d-4ffe-9cca-9b7ca0f5c329 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:05:16.094443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00230f19-640d-4004-a293-e70a23c56748 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:05:16.095174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00230f19-640d-4004-a293-e70a23c56748 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:16.095500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00230f19-640d-4004-a293-e70a23c56748 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:16.095886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00230f19-640d-4004-a293-e70a23c56748 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:16.106092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00230f19-640d-4004-a293-e70a23c56748 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:05:16.107847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00230f19-640d-4004-a293-e70a23c56748 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:05:16.119663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 752/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:05:17.259838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e89d83-aaf8-47a8-8c62-b56eb583faaa None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:17.260723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e89d83-aaf8-47a8-8c62-b56eb583faaa None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:17.260905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e89d83-aaf8-47a8-8c62-b56eb583faaa None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:17.262391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 17:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:05:17.270458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d2a4-3705-4604-ace5-bbd1037b3271 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:17.271142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d2a4-3705-4604-ace5-bbd1037b3271 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:17.271142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d2a4-3705-4604-ace5-bbd1037b3271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:17.293583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2de0d2a4-3705-4604-ace5-bbd1037b3271 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:05:17.294843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2de0d2a4-3705-4604-ace5-bbd1037b3271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:17.304693 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 17:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:05:17.460343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e89d83-aaf8-47a8-8c62-b56eb583faaa None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:05:17.476327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e89d83-aaf8-47a8-8c62-b56eb583faaa None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:05:17.482612 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c287c-0587-4113-bb51-9e37b9224bac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:05:17.483304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c287c-0587-4113-bb51-9e37b9224bac service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:17.483586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c287c-0587-4113-bb51-9e37b9224bac service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:17.483858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c287c-0587-4113-bb51-9e37b9224bac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:17.493843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c287c-0587-4113-bb51-9e37b9224bac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:05:17.495502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c287c-0587-4113-bb51-9e37b9224bac service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:05:17.506543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 774/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:05:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:05:22.067548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2559-c49e-41d3-b4a5-47ac28413f47 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:22.068431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2559-c49e-41d3-b4a5-47ac28413f47 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:22.068431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2559-c49e-41d3-b4a5-47ac28413f47 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:22.069875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:05:22.075747 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305405e3-0997-4a3d-b809-07d4f860f4c3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:22.076323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305405e3-0997-4a3d-b809-07d4f860f4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:22.076323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305405e3-0997-4a3d-b809-07d4f860f4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:22.098145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-305405e3-0997-4a3d-b809-07d4f860f4c3 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:05:22.099399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-305405e3-0997-4a3d-b809-07d4f860f4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:22.107474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:05:22.403274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c2559-c49e-41d3-b4a5-47ac28413f47 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:05:22.405121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-381c2559-c49e-41d3-b4a5-47ac28413f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:22.472634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c2559-c49e-41d3-b4a5-47ac28413f47 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:05:22.476934 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783df22-ad3e-4a2c-9764-508068861467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:05:22.477563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783df22-ad3e-4a2c-9764-508068861467 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:22.477845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783df22-ad3e-4a2c-9764-508068861467 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:22.478117 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783df22-ad3e-4a2c-9764-508068861467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:22.488869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783df22-ad3e-4a2c-9764-508068861467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:05:22.491998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783df22-ad3e-4a2c-9764-508068861467 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:05:22.509957 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 755/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:05:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:05:24.793473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b90cd-c149-4f7d-b07d-0cc507613a2f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:24.794068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b90cd-c149-4f7d-b07d-0cc507613a2f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:24.794068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b90cd-c149-4f7d-b07d-0cc507613a2f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:24.795660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:05:24.802846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb70c84-3fb3-444f-8fc8-926b24d175e9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:24.803505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb70c84-3fb3-444f-8fc8-926b24d175e9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:24.803505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb70c84-3fb3-444f-8fc8-926b24d175e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:24.828809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-edb70c84-3fb3-444f-8fc8-926b24d175e9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:05:24.830819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-edb70c84-3fb3-444f-8fc8-926b24d175e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:24.844602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:05:25.138698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b90cd-c149-4f7d-b07d-0cc507613a2f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:05:25.140697 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c95b90cd-c149-4f7d-b07d-0cc507613a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:25.155140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b90cd-c149-4f7d-b07d-0cc507613a2f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:05:25.158856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11b085-767b-4566-ad77-c78c0d6e126d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:05:25.159493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11b085-767b-4566-ad77-c78c0d6e126d service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:25.159768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11b085-767b-4566-ad77-c78c0d6e126d service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:25.160055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11b085-767b-4566-ad77-c78c0d6e126d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:25.170049 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11b085-767b-4566-ad77-c78c0d6e126d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:05:25.171531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11b085-767b-4566-ad77-c78c0d6e126d service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:05:25.182526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 777/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:05:29.077775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a2be9-ce44-4e53-8137-9d411bf03442 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:29.078337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a2be9-ce44-4e53-8137-9d411bf03442 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:29.078337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a2be9-ce44-4e53-8137-9d411bf03442 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:29.079869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:05:29.087382 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3407c9a-d053-43f4-8cc0-e958b472a8a4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:29.088083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3407c9a-d053-43f4-8cc0-e958b472a8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:29.088083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3407c9a-d053-43f4-8cc0-e958b472a8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:29.109552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c3407c9a-d053-43f4-8cc0-e958b472a8a4 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:05:29.110784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c3407c9a-d053-43f4-8cc0-e958b472a8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:29.118926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:05:29.428393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a2be9-ce44-4e53-8137-9d411bf03442 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:05:29.430106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c24a2be9-ce44-4e53-8137-9d411bf03442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:29.436323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=75 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:05:29.498566 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a2be9-ce44-4e53-8137-9d411bf03442 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:05:29.502794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad5578-a7e9-4f53-b789-66404ca0ff66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:05:29.503449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad5578-a7e9-4f53-b789-66404ca0ff66 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:29.503729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad5578-a7e9-4f53-b789-66404ca0ff66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:29.504005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad5578-a7e9-4f53-b789-66404ca0ff66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:29.516286 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad5578-a7e9-4f53-b789-66404ca0ff66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:05:29.519648 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad5578-a7e9-4f53-b789-66404ca0ff66 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:05:29.539566 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 758/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:05:31.804551 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f5ae-753b-4d6a-acf3-dbc9cd14a3c4 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:31.805135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f5ae-753b-4d6a-acf3-dbc9cd14a3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:31.805135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f5ae-753b-4d6a-acf3-dbc9cd14a3c4 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:31.806448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 17:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:05:31.812598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e985dd9-78a0-40c9-9f30-dac4fb3f1e81 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:31.813171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e985dd9-78a0-40c9-9f30-dac4fb3f1e81 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:31.813280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e985dd9-78a0-40c9-9f30-dac4fb3f1e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:31.835936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2e985dd9-78a0-40c9-9f30-dac4fb3f1e81 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:05:31.837320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2e985dd9-78a0-40c9-9f30-dac4fb3f1e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:31.846449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 17:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:05:32.164258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8f5ae-753b-4d6a-acf3-dbc9cd14a3c4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:05:32.166195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d2c8f5ae-753b-4d6a-acf3-dbc9cd14a3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:05:32.180336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8f5ae-753b-4d6a-acf3-dbc9cd14a3c4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:05:32.184528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b9de4-c707-4e3d-be57-72527ce8f258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:05:32.185272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9de4-c707-4e3d-be57-72527ce8f258 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:05:32.185663 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9de4-c707-4e3d-be57-72527ce8f258 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:05:32.186031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9de4-c707-4e3d-be57-72527ce8f258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:05:32.198181 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b9de4-c707-4e3d-be57-72527ce8f258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:05:32.199877 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b9de4-c707-4e3d-be57-72527ce8f258 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:05:32.211600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 780/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:05:41.828743 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=28 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:05:42.206617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=26 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:00.193266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56282466-a4f0-40bd-aa41-6b564f4cd9a6 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:00.195060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56282466-a4f0-40bd-aa41-6b564f4cd9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:00.195874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56282466-a4f0-40bd-aa41-6b564f4cd9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:00.244278 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-56282466-a4f0-40bd-aa41-6b564f4cd9a6 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:00.247030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-56282466-a4f0-40bd-aa41-6b564f4cd9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:00.267226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 760/1540] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:00.276007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5f8b7-f012-4fd2-adb9-582b60019d6c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:00.277071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5f8b7-f012-4fd2-adb9-582b60019d6c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:00.277593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5f8b7-f012-4fd2-adb9-582b60019d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:00.309762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a6f5f8b7-f012-4fd2-adb9-582b60019d6c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:00.312049 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a6f5f8b7-f012-4fd2-adb9-582b60019d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:00.323824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 781/1541] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:00.329541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56282466-a4f0-40bd-aa41-6b564f4cd9a6 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:00.358208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56282466-a4f0-40bd-aa41-6b564f4cd9a6 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:00.374919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6663926-4420-4707-b2cd-b0a8144190ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:00.376311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6663926-4420-4707-b2cd-b0a8144190ef admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:00.376819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6663926-4420-4707-b2cd-b0a8144190ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:00.377273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6663926-4420-4707-b2cd-b0a8144190ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:00.378499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6663926-4420-4707-b2cd-b0a8144190ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:00.379062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6663926-4420-4707-b2cd-b0a8144190ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:00.382501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6663926-4420-4707-b2cd-b0a8144190ef admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:00.417887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 761/1542] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:00.425983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5f8b7-f012-4fd2-adb9-582b60019d6c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:00.440739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5f8b7-f012-4fd2-adb9-582b60019d6c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:00.446606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee38e4-b38b-4fc1-8a85-dc54077759d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:00.447630 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee38e4-b38b-4fc1-8a85-dc54077759d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:00.448141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee38e4-b38b-4fc1-8a85-dc54077759d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:00.448606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee38e4-b38b-4fc1-8a85-dc54077759d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:00.452769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee38e4-b38b-4fc1-8a85-dc54077759d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:00.455512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee38e4-b38b-4fc1-8a85-dc54077759d3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:00.494725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 782/1543] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:00 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:00.501095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6663926-4420-4707-b2cd-b0a8144190ef admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:00.526698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6663926-4420-4707-b2cd-b0a8144190ef admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:00.633581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7a9b5-ca6e-4bc2-9a4f-417e0d827761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:00.634629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7a9b5-ca6e-4bc2-9a4f-417e0d827761 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:00.635162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7a9b5-ca6e-4bc2-9a4f-417e0d827761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:00.635680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7a9b5-ca6e-4bc2-9a4f-417e0d827761 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:00.636561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7a9b5-ca6e-4bc2-9a4f-417e0d827761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:00.638936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7a9b5-ca6e-4bc2-9a4f-417e0d827761 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:00.678758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 762/1544] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:00 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:00.683597 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee38e4-b38b-4fc1-8a85-dc54077759d3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:00.698722 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee38e4-b38b-4fc1-8a85-dc54077759d3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:00.707525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ce338-bc93-461e-a9cd-90816308f1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:00.708777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ce338-bc93-461e-a9cd-90816308f1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:00.709306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ce338-bc93-461e-a9cd-90816308f1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:00.709833 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ce338-bc93-461e-a9cd-90816308f1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:00.710742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ce338-bc93-461e-a9cd-90816308f1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:00.712743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ce338-bc93-461e-a9cd-90816308f1d9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:00.730475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 783/1545] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:00.735232 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7a9b5-ca6e-4bc2-9a4f-417e0d827761 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:00.771035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7a9b5-ca6e-4bc2-9a4f-417e0d827761 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:00.784339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad3031-841b-42cb-b12c-72171cceed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:00.785773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad3031-841b-42cb-b12c-72171cceed61 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:00.786261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad3031-841b-42cb-b12c-72171cceed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:00.786769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad3031-841b-42cb-b12c-72171cceed61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:00.788035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad3031-841b-42cb-b12c-72171cceed61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:00.791300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad3031-841b-42cb-b12c-72171cceed61 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:00.821453 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 763/1546] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:00.826749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ce338-bc93-461e-a9cd-90816308f1d9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:00.863197 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ce338-bc93-461e-a9cd-90816308f1d9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:00.880279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59139ac-bb6f-4cb5-ac16-6605bc474343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:00.881448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59139ac-bb6f-4cb5-ac16-6605bc474343 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:00.881954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59139ac-bb6f-4cb5-ac16-6605bc474343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:00.882433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59139ac-bb6f-4cb5-ac16-6605bc474343 admin admin] PATH_INFO: `/v3/projects/d992b97fcee74a69baacab0b30f11d57/users/e94051ebd36148e890cb9e75d98eb579/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:00.897638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fdb99-e6fb-4309-9dc9-213b0a9b0450 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:00.898100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fdb99-e6fb-4309-9dc9-213b0a9b0450 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:00.898544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fdb99-e6fb-4309-9dc9-213b0a9b0450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:00.928403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59139ac-bb6f-4cb5-ac16-6605bc474343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d992b97fcee74a69baacab0b30f11d57, user_id=e94051ebd36148e890cb9e75d98eb579, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:00.933604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59139ac-bb6f-4cb5-ac16-6605bc474343 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:00.969845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bb1fdb99-e6fb-4309-9dc9-213b0a9b0450 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:00.972040 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bb1fdb99-e6fb-4309-9dc9-213b0a9b0450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:00.999706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 784/1547] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:00 2022] PUT /identity/v3/projects/d992b97fcee74a69baacab0b30f11d57/users/e94051ebd36148e890cb9e75d98eb579/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:01.013729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d210e-0a6e-4107-a875-f168183c9ab3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.013729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d210e-0a6e-4107-a875-f168183c9ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.014227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d210e-0a6e-4107-a875-f168183c9ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.114975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-cc5d210e-0a6e-4107-a875-f168183c9ab3 None None] MFA Rules not processed for user `e94051ebd36148e890cb9e75d98eb579`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.117267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cc5d210e-0a6e-4107-a875-f168183c9ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.162623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 764/1548] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:01.170563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87b0e9-1d6a-40d3-88be-29881b42caec None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.171090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87b0e9-1d6a-40d3-88be-29881b42caec None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.171543 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87b0e9-1d6a-40d3-88be-29881b42caec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.236324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-dd87b0e9-1d6a-40d3-88be-29881b42caec None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.238973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-dd87b0e9-1d6a-40d3-88be-29881b42caec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.265347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 765/1549] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:01.267523 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80cb9f-87af-48ea-be02-646c66446079 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.267982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80cb9f-87af-48ea-be02-646c66446079 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.268424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80cb9f-87af-48ea-be02-646c66446079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.320848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-de80cb9f-87af-48ea-be02-646c66446079 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.327296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-de80cb9f-87af-48ea-be02-646c66446079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.348116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 766/1550] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:01.349646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80cb9f-87af-48ea-be02-646c66446079 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:01.380290 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 785/1551] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:01.383550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c638-dc2a-48f2-a845-6738eb7d02ff None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.386214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c638-dc2a-48f2-a845-6738eb7d02ff None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.386749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c638-dc2a-48f2-a845-6738eb7d02ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.395401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80cb9f-87af-48ea-be02-646c66446079 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:01.409741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80126b2-cbd7-4c49-aa7f-5901c206c9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:01.410813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80126b2-cbd7-4c49-aa7f-5901c206c9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.411335 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80126b2-cbd7-4c49-aa7f-5901c206c9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.411797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80126b2-cbd7-4c49-aa7f-5901c206c9f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.412990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80126b2-cbd7-4c49-aa7f-5901c206c9f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:01.413529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80126b2-cbd7-4c49-aa7f-5901c206c9f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:01.416670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80126b2-cbd7-4c49-aa7f-5901c206c9f8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:01.455180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 767/1552] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:01.458363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565bfcb-8697-4ec1-bf7d-44852dbd210f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.458790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565bfcb-8697-4ec1-bf7d-44852dbd210f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.459268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565bfcb-8697-4ec1-bf7d-44852dbd210f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.471780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-43f4c638-dc2a-48f2-a845-6738eb7d02ff None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.474454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-43f4c638-dc2a-48f2-a845-6738eb7d02ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.487208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 786/1553] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:01.488623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032130e-f58e-4e2a-9002-95255e731593 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.488924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032130e-f58e-4e2a-9002-95255e731593 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.489225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032130e-f58e-4e2a-9002-95255e731593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.512873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8565bfcb-8697-4ec1-bf7d-44852dbd210f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.513349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c032130e-f58e-4e2a-9002-95255e731593 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.514804 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8565bfcb-8697-4ec1-bf7d-44852dbd210f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.515308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c032130e-f58e-4e2a-9002-95255e731593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.525437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 787/1554] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:01.526309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032130e-f58e-4e2a-9002-95255e731593 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:01.526640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 768/1555] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:01.527492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565bfcb-8697-4ec1-bf7d-44852dbd210f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:01.541026 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565bfcb-8697-4ec1-bf7d-44852dbd210f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:01.546612 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032130e-f58e-4e2a-9002-95255e731593 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:01.547189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc235b-e7f7-43ad-acaf-8aed0ab983c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:01.547956 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc235b-e7f7-43ad-acaf-8aed0ab983c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.549309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc235b-e7f7-43ad-acaf-8aed0ab983c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.549994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc235b-e7f7-43ad-acaf-8aed0ab983c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.551743 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc235b-e7f7-43ad-acaf-8aed0ab983c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:01.553110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-badc235b-e7f7-43ad-acaf-8aed0ab983c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:07:01.553110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 22 17:07:01.553110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: Sep 22 17:07:01.553859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d145145-5978-45ff-b7a3-32ada69e5df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:01.554431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc235b-e7f7-43ad-acaf-8aed0ab983c6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:01.554592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d145145-5978-45ff-b7a3-32ada69e5df9 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.554875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d145145-5978-45ff-b7a3-32ada69e5df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.555236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d145145-5978-45ff-b7a3-32ada69e5df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.555841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d145145-5978-45ff-b7a3-32ada69e5df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:01.558051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d145145-5978-45ff-b7a3-32ada69e5df9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:01.599678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 769/1556] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:01.601619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 788/1557] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/projects => generated 363 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:01.602965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4b54b-d4e9-484f-be5b-93e77052cea8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.603349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4b54b-d4e9-484f-be5b-93e77052cea8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.603505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4b54b-d4e9-484f-be5b-93e77052cea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.604740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89344e-424f-4674-aa62-a22d10ad529f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.605034 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89344e-424f-4674-aa62-a22d10ad529f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.607646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89344e-424f-4674-aa62-a22d10ad529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.624820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:01.631124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-87b4b54b-d4e9-484f-be5b-93e77052cea8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.632551 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-87b4b54b-d4e9-484f-be5b-93e77052cea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.643091 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-fa89344e-424f-4674-aa62-a22d10ad529f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.644385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fa89344e-424f-4674-aa62-a22d10ad529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.694432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:01.717714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 789/1558] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:01.718504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4b54b-d4e9-484f-be5b-93e77052cea8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:01.730656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4b54b-d4e9-484f-be5b-93e77052cea8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:01.736237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3ccac-a35f-4602-ba8a-dc2f3c905801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:01.736822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3ccac-a35f-4602-ba8a-dc2f3c905801 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.737165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3ccac-a35f-4602-ba8a-dc2f3c905801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.737617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3ccac-a35f-4602-ba8a-dc2f3c905801 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.738151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3ccac-a35f-4602-ba8a-dc2f3c905801 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:01.738458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3ccac-a35f-4602-ba8a-dc2f3c905801 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:01.743428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 770/1559] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:01.743657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-27c3ccac-a35f-4602-ba8a-dc2f3c905801 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:07:01.743657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Sep 22 17:07:01.743657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: Sep 22 17:07:01.743657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3ccac-a35f-4602-ba8a-dc2f3c905801 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:01.744175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b51fb-0302-4ffa-b139-7462761f2791 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.744175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b51fb-0302-4ffa-b139-7462761f2791 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.744175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b51fb-0302-4ffa-b139-7462761f2791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.765692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 790/1560] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:01.765692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3ccac-a35f-4602-ba8a-dc2f3c905801 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:01.782060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3ccac-a35f-4602-ba8a-dc2f3c905801 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:01.793043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155da40-af6e-4c82-9a7f-705efb880c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:01.798397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155da40-af6e-4c82-9a7f-705efb880c34 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.798397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155da40-af6e-4c82-9a7f-705efb880c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.798397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155da40-af6e-4c82-9a7f-705efb880c34 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.798397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155da40-af6e-4c82-9a7f-705efb880c34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:01.800620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-e155da40-af6e-4c82-9a7f-705efb880c34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:07:01.800620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Sep 22 17:07:01.800620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: Sep 22 17:07:01.803238 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155da40-af6e-4c82-9a7f-705efb880c34 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:01.805145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ac7b51fb-0302-4ffa-b139-7462761f2791 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.807341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ac7b51fb-0302-4ffa-b139-7462761f2791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.821505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 771/1561] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:01.823211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b51fb-0302-4ffa-b139-7462761f2791 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:01.843446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b51fb-0302-4ffa-b139-7462761f2791 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:01.847143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 791/1562] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/users => generated 289 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:01.847394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df1754-ffa1-4f58-8499-a4920e247858 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.847866 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df1754-ffa1-4f58-8499-a4920e247858 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.848332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df1754-ffa1-4f58-8499-a4920e247858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.854370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cbd51-0dc3-4e5e-9757-c469c9dd37ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:01.855495 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cbd51-0dc3-4e5e-9757-c469c9dd37ae admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.855982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cbd51-0dc3-4e5e-9757-c469c9dd37ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.856430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cbd51-0dc3-4e5e-9757-c469c9dd37ae admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.857627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cbd51-0dc3-4e5e-9757-c469c9dd37ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:01.861593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cbd51-0dc3-4e5e-9757-c469c9dd37ae admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:01.882500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-88df1754-ffa1-4f58-8499-a4920e247858 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.884658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-88df1754-ffa1-4f58-8499-a4920e247858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.896117 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 772/1563] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/users => generated 288 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:01.896117 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959fdbd-6940-4651-ae75-98996a0aceee None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.896117 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959fdbd-6940-4651-ae75-98996a0aceee None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.896117 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959fdbd-6940-4651-ae75-98996a0aceee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.907017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 792/1564] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:01.907017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df1754-ffa1-4f58-8499-a4920e247858 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:01.918105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2959fdbd-6940-4651-ae75-98996a0aceee None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.918348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df1754-ffa1-4f58-8499-a4920e247858 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:01.919500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2959fdbd-6940-4651-ae75-98996a0aceee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.927046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd041a-dfef-487c-8f20-ce04e8708858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:01.928086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd041a-dfef-487c-8f20-ce04e8708858 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.928396 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 773/1565] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:01.928580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd041a-dfef-487c-8f20-ce04e8708858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.929054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd041a-dfef-487c-8f20-ce04e8708858 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.929316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959fdbd-6940-4651-ae75-98996a0aceee None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:01.930095 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd041a-dfef-487c-8f20-ce04e8708858 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:01.930529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd041a-dfef-487c-8f20-ce04e8708858 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:01.933151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd041a-dfef-487c-8f20-ce04e8708858 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:01.940839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959fdbd-6940-4651-ae75-98996a0aceee None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:01.945996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e0eb7-f7c5-415f-8fbc-cfe3f4cc532f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:01.947415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e0eb7-f7c5-415f-8fbc-cfe3f4cc532f admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.947722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e0eb7-f7c5-415f-8fbc-cfe3f4cc532f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.948004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e0eb7-f7c5-415f-8fbc-cfe3f4cc532f admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.948666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e0eb7-f7c5-415f-8fbc-cfe3f4cc532f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:01.951414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e0eb7-f7c5-415f-8fbc-cfe3f4cc532f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:01.955912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 793/1566] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:01.958128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc96fdb-fdc2-45bc-83ec-ad5ff42813f8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:01.958128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc96fdb-fdc2-45bc-83ec-ad5ff42813f8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:01.958557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc96fdb-fdc2-45bc-83ec-ad5ff42813f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:01.990253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7fc96fdb-fdc2-45bc-83ec-ad5ff42813f8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:01.991482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7fc96fdb-fdc2-45bc-83ec-ad5ff42813f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:01.992266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 774/1567] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:01.993674 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e0eb7-f7c5-415f-8fbc-cfe3f4cc532f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.065031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 794/1568] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:02.066363 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc96fdb-fdc2-45bc-83ec-ad5ff42813f8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.079536 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc96fdb-fdc2-45bc-83ec-ad5ff42813f8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.084781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7cbb-be64-4281-8d37-ca8cd242473b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.085473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7cbb-be64-4281-8d37-ca8cd242473b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.085740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7cbb-be64-4281-8d37-ca8cd242473b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.086069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7cbb-be64-4281-8d37-ca8cd242473b admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.086792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d7cbb-be64-4281-8d37-ca8cd242473b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.088656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d7cbb-be64-4281-8d37-ca8cd242473b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.096571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e0eb7-f7c5-415f-8fbc-cfe3f4cc532f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.100310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 795/1569] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:02.101431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7cbb-be64-4281-8d37-ca8cd242473b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.104836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939273d-7dc1-441b-a953-1f0d5cd9697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.105634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939273d-7dc1-441b-a953-1f0d5cd9697e admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.106020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939273d-7dc1-441b-a953-1f0d5cd9697e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.106398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939273d-7dc1-441b-a953-1f0d5cd9697e admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.107477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939273d-7dc1-441b-a953-1f0d5cd9697e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:02.108574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939273d-7dc1-441b-a953-1f0d5cd9697e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.110786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939273d-7dc1-441b-a953-1f0d5cd9697e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.113672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7cbb-be64-4281-8d37-ca8cd242473b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.119159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41374b82-1691-4bdc-b9e1-197bab5cb79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.119794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41374b82-1691-4bdc-b9e1-197bab5cb79b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.120116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41374b82-1691-4bdc-b9e1-197bab5cb79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.120419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41374b82-1691-4bdc-b9e1-197bab5cb79b admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.121113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41374b82-1691-4bdc-b9e1-197bab5cb79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.122884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41374b82-1691-4bdc-b9e1-197bab5cb79b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.132248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 775/1570] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:02.134734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae0f5d-14db-4b63-87f3-277606fb43b1 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.135272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae0f5d-14db-4b63-87f3-277606fb43b1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.135754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae0f5d-14db-4b63-87f3-277606fb43b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.138105 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 796/1571] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:02.140997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41374b82-1691-4bdc-b9e1-197bab5cb79b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.155077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41374b82-1691-4bdc-b9e1-197bab5cb79b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.160177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-69ae0f5d-14db-4b63-87f3-277606fb43b1 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:02.160842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b46685-195f-405e-8cde-28ee32e1a654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.161434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-69ae0f5d-14db-4b63-87f3-277606fb43b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:02.161682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b46685-195f-405e-8cde-28ee32e1a654 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.162063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b46685-195f-405e-8cde-28ee32e1a654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.162512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b46685-195f-405e-8cde-28ee32e1a654 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.163458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b46685-195f-405e-8cde-28ee32e1a654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:02.163870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b46685-195f-405e-8cde-28ee32e1a654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.165614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b46685-195f-405e-8cde-28ee32e1a654 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.170172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 776/1572] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:02.175943 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae0f5d-14db-4b63-87f3-277606fb43b1 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.182620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 797/1573] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:02.183877 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b46685-195f-405e-8cde-28ee32e1a654 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.191414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae0f5d-14db-4b63-87f3-277606fb43b1 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.198072 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016cc910-ce43-4f0e-9e65-aab2dc96642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.199146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cc910-ce43-4f0e-9e65-aab2dc96642b admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.199608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cc910-ce43-4f0e-9e65-aab2dc96642b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.200060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016cc910-ce43-4f0e-9e65-aab2dc96642b admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.201189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016cc910-ce43-4f0e-9e65-aab2dc96642b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.201357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b46685-195f-405e-8cde-28ee32e1a654 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.206148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016cc910-ce43-4f0e-9e65-aab2dc96642b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.206636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ea496-fd01-41e0-8c5e-cbfc31255da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.207286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ea496-fd01-41e0-8c5e-cbfc31255da3 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.207566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ea496-fd01-41e0-8c5e-cbfc31255da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.207840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ea496-fd01-41e0-8c5e-cbfc31255da3 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.208488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ea496-fd01-41e0-8c5e-cbfc31255da3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.210441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ea496-fd01-41e0-8c5e-cbfc31255da3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.236379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 798/1574] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:02 2022] POST /identity/v3/users => generated 287 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:02.237755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589c12b-df1d-4b1e-85bd-d9f43787f903 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.238111 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589c12b-df1d-4b1e-85bd-d9f43787f903 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.238432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589c12b-df1d-4b1e-85bd-d9f43787f903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.256252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 777/1575] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:02 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:02.257654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016cc910-ce43-4f0e-9e65-aab2dc96642b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.264459 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f589c12b-df1d-4b1e-85bd-d9f43787f903 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:02.266045 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f589c12b-df1d-4b1e-85bd-d9f43787f903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:02.287195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:02.333466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:02.346776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016cc910-ce43-4f0e-9e65-aab2dc96642b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.351799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4e7e2-614a-444b-ba9f-0bd6cc276f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.352429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4e7e2-614a-444b-ba9f-0bd6cc276f8c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.352712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4e7e2-614a-444b-ba9f-0bd6cc276f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.352976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4e7e2-614a-444b-ba9f-0bd6cc276f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.353628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4e7e2-614a-444b-ba9f-0bd6cc276f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.355354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4e7e2-614a-444b-ba9f-0bd6cc276f8c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.366594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 778/1576] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:02.367523 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4e7e2-614a-444b-ba9f-0bd6cc276f8c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.367962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 799/1577] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:02.369728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589c12b-df1d-4b1e-85bd-d9f43787f903 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.384930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589c12b-df1d-4b1e-85bd-d9f43787f903 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.385632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4e7e2-614a-444b-ba9f-0bd6cc276f8c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.390616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120788d-c728-43f6-924f-2c0428d076ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.390817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e24c9-c022-439a-8d4e-2e29d1b47405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.391375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120788d-c728-43f6-924f-2c0428d076ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.391492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e24c9-c022-439a-8d4e-2e29d1b47405 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.391659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120788d-c728-43f6-924f-2c0428d076ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.391771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e24c9-c022-439a-8d4e-2e29d1b47405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.391937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e24c9-c022-439a-8d4e-2e29d1b47405 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.392048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120788d-c728-43f6-924f-2c0428d076ca admin admin] PATH_INFO: `/v3/projects/68ee1f457ed84904b5b4a85a1412ecdc/users/4332f4e1592840e2bd191fafb8c6944b/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.392579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e24c9-c022-439a-8d4e-2e29d1b47405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.394532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e24c9-c022-439a-8d4e-2e29d1b47405 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.411713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120788d-c728-43f6-924f-2c0428d076ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ee1f457ed84904b5b4a85a1412ecdc, user_id=4332f4e1592840e2bd191fafb8c6944b, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.414775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-5120788d-c728-43f6-924f-2c0428d076ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:07:02.414775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 22 17:07:02.414775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: Sep 22 17:07:02.419270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120788d-c728-43f6-924f-2c0428d076ca admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.429971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 800/1578] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:02 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:02.431492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e24c9-c022-439a-8d4e-2e29d1b47405 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.442086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 779/1579] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:02 2022] PUT /identity/v3/projects/68ee1f457ed84904b5b4a85a1412ecdc/users/4332f4e1592840e2bd191fafb8c6944b/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:02.443388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120788d-c728-43f6-924f-2c0428d076ca admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.507630 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e24c9-c022-439a-8d4e-2e29d1b47405 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.554961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120788d-c728-43f6-924f-2c0428d076ca admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.563120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385720d8-2365-4c3f-bc3f-3fa887ec952a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.563883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385720d8-2365-4c3f-bc3f-3fa887ec952a admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.564260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385720d8-2365-4c3f-bc3f-3fa887ec952a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.564630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385720d8-2365-4c3f-bc3f-3fa887ec952a admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.565367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385720d8-2365-4c3f-bc3f-3fa887ec952a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.567676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385720d8-2365-4c3f-bc3f-3fa887ec952a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.587397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875797d-5cbb-40fd-a3d4-e37933e708e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.588343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875797d-5cbb-40fd-a3d4-e37933e708e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.588777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875797d-5cbb-40fd-a3d4-e37933e708e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.589151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875797d-5cbb-40fd-a3d4-e37933e708e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.590183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875797d-5cbb-40fd-a3d4-e37933e708e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:02.590669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875797d-5cbb-40fd-a3d4-e37933e708e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.594036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875797d-5cbb-40fd-a3d4-e37933e708e4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.612511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 780/1580] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:02 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:02.622601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 801/1581] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:02.622918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385720d8-2365-4c3f-bc3f-3fa887ec952a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.623113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875797d-5cbb-40fd-a3d4-e37933e708e4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.636385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875797d-5cbb-40fd-a3d4-e37933e708e4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.709607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385720d8-2365-4c3f-bc3f-3fa887ec952a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.715980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9f756-eed6-4893-8cde-ad02cd440e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.716725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9f756-eed6-4893-8cde-ad02cd440e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.717118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9f756-eed6-4893-8cde-ad02cd440e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.717564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a9828-21cf-4033-b738-2ef0e28ac3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.717754 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9f756-eed6-4893-8cde-ad02cd440e57 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.718525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9f756-eed6-4893-8cde-ad02cd440e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.718711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9828-21cf-4033-b738-2ef0e28ac3cb admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.719174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9828-21cf-4033-b738-2ef0e28ac3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.719656 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9828-21cf-4033-b738-2ef0e28ac3cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.720613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a9828-21cf-4033-b738-2ef0e28ac3cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.721233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9f756-eed6-4893-8cde-ad02cd440e57 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.723675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a9828-21cf-4033-b738-2ef0e28ac3cb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.740755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:02.748281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 781/1582] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:02.748281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a9828-21cf-4033-b738-2ef0e28ac3cb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.757424 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 802/1583] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:02 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:02.759612 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9f756-eed6-4893-8cde-ad02cd440e57 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.764549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a9828-21cf-4033-b738-2ef0e28ac3cb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.771183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9f756-eed6-4893-8cde-ad02cd440e57 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.772651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd0040-c23e-49ea-88d1-62c06d226384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.773707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd0040-c23e-49ea-88d1-62c06d226384 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.774206 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd0040-c23e-49ea-88d1-62c06d226384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.774646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd0040-c23e-49ea-88d1-62c06d226384 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.775872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd0040-c23e-49ea-88d1-62c06d226384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:02.776417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd0040-c23e-49ea-88d1-62c06d226384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.776715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c66c34-c831-426f-97d0-47d417126916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.777390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c66c34-c831-426f-97d0-47d417126916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.777667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c66c34-c831-426f-97d0-47d417126916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.777936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c66c34-c831-426f-97d0-47d417126916 admin admin] PATH_INFO: `/v3/projects/70b4f8716fa84e439293149dd10cac82/users/3725446c6d5040c7870b056d4bfbeca6/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.779520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd0040-c23e-49ea-88d1-62c06d226384 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.788843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:02.800682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c66c34-c831-426f-97d0-47d417126916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70b4f8716fa84e439293149dd10cac82, user_id=3725446c6d5040c7870b056d4bfbeca6, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.801466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 782/1584] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:02.802806 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd0040-c23e-49ea-88d1-62c06d226384 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.811279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c66c34-c831-426f-97d0-47d417126916 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.821352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd0040-c23e-49ea-88d1-62c06d226384 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.830233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 803/1585] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:02 2022] PUT /identity/v3/projects/70b4f8716fa84e439293149dd10cac82/users/3725446c6d5040c7870b056d4bfbeca6/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:02.831720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c66c34-c831-426f-97d0-47d417126916 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.917831 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadef56-2cbb-4ca7-93a2-57e4ccebdd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.918537 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadef56-2cbb-4ca7-93a2-57e4ccebdd8e admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.918999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadef56-2cbb-4ca7-93a2-57e4ccebdd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.919409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadef56-2cbb-4ca7-93a2-57e4ccebdd8e admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.920119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadef56-2cbb-4ca7-93a2-57e4ccebdd8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.923210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadef56-2cbb-4ca7-93a2-57e4ccebdd8e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.954411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 783/1586] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:02 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:02.954411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadef56-2cbb-4ca7-93a2-57e4ccebdd8e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.954900 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c66c34-c831-426f-97d0-47d417126916 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.959455 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7380b-c261-4ee4-b271-c32d80796c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.960530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7380b-c261-4ee4-b271-c32d80796c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.961024 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7380b-c261-4ee4-b271-c32d80796c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.961514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7380b-c261-4ee4-b271-c32d80796c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.962724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7380b-c261-4ee4-b271-c32d80796c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.964729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadef56-2cbb-4ca7-93a2-57e4ccebdd8e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:02.965887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7380b-c261-4ee4-b271-c32d80796c6f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.969649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304f979-1e51-465c-b57b-3880f6ae95e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:02.970272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304f979-1e51-465c-b57b-3880f6ae95e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:02.970551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304f979-1e51-465c-b57b-3880f6ae95e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:02.970824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304f979-1e51-465c-b57b-3880f6ae95e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:02.971507 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304f979-1e51-465c-b57b-3880f6ae95e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:02.973455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304f979-1e51-465c-b57b-3880f6ae95e7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:02.981718 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 804/1587] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:02.983573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7380b-c261-4ee4-b271-c32d80796c6f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:02.996462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7380b-c261-4ee4-b271-c32d80796c6f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.001457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 784/1588] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:02 2022] POST /identity/v3/users => generated 287 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:03.002612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304f979-1e51-465c-b57b-3880f6ae95e7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.002996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5ad6a-accc-4d5b-b4b1-99604078c5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.005269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ad6a-accc-4d5b-b4b1-99604078c5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.006075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ad6a-accc-4d5b-b4b1-99604078c5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.006689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ad6a-accc-4d5b-b4b1-99604078c5f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.007695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5ad6a-accc-4d5b-b4b1-99604078c5f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.010425 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5ad6a-accc-4d5b-b4b1-99604078c5f2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.016125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304f979-1e51-465c-b57b-3880f6ae95e7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.022109 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2779cc1-9877-4f0c-a283-5d4e5232f13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.022819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2779cc1-9877-4f0c-a283-5d4e5232f13b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.023926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2779cc1-9877-4f0c-a283-5d4e5232f13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.024367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2779cc1-9877-4f0c-a283-5d4e5232f13b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.025200 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2779cc1-9877-4f0c-a283-5d4e5232f13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.027283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2779cc1-9877-4f0c-a283-5d4e5232f13b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.046152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 785/1589] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:03.046152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2779cc1-9877-4f0c-a283-5d4e5232f13b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.053223 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 805/1590] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:02 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:03.054236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5ad6a-accc-4d5b-b4b1-99604078c5f2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.060290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2779cc1-9877-4f0c-a283-5d4e5232f13b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.131241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5ad6a-accc-4d5b-b4b1-99604078c5f2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.136131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16015bfc-ba39-4ae5-a84d-a7900256d288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.136391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d2851-4995-4cf8-80f8-1075f59c9641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.136865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16015bfc-ba39-4ae5-a84d-a7900256d288 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.137042 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d2851-4995-4cf8-80f8-1075f59c9641 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.137242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d2851-4995-4cf8-80f8-1075f59c9641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.137398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16015bfc-ba39-4ae5-a84d-a7900256d288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.137585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d2851-4995-4cf8-80f8-1075f59c9641 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.137765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16015bfc-ba39-4ae5-a84d-a7900256d288 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.138049 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d2851-4995-4cf8-80f8-1075f59c9641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.138438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16015bfc-ba39-4ae5-a84d-a7900256d288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.140103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d2851-4995-4cf8-80f8-1075f59c9641 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.140931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16015bfc-ba39-4ae5-a84d-a7900256d288 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.158629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 786/1591] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:03.159937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd79f8a-33f9-41f6-952e-2471ef6cb9e7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.160275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd79f8a-33f9-41f6-952e-2471ef6cb9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.160529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd79f8a-33f9-41f6-952e-2471ef6cb9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.183484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 806/1592] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:03 2022] POST /identity/v3/projects => generated 391 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:03.186578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6bd79f8a-33f9-41f6-952e-2471ef6cb9e7 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:03.187878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6bd79f8a-33f9-41f6-952e-2471ef6cb9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:03.188360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d2851-4995-4cf8-80f8-1075f59c9641 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.244313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:03.258835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:03.274487 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 787/1593] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:03.275483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd79f8a-33f9-41f6-952e-2471ef6cb9e7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.286780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd79f8a-33f9-41f6-952e-2471ef6cb9e7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.292224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ba37e-a0bd-4f00-ac77-70668a9d0e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.292963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ba37e-a0bd-4f00-ac77-70668a9d0e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.293239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ba37e-a0bd-4f00-ac77-70668a9d0e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.293519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ba37e-a0bd-4f00-ac77-70668a9d0e17 admin admin] PATH_INFO: `/v3/projects/68ee1f457ed84904b5b4a85a1412ecdc/users/4332f4e1592840e2bd191fafb8c6944b/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.295711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ba37e-a0bd-4f00-ac77-70668a9d0e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ee1f457ed84904b5b4a85a1412ecdc, user_id=4332f4e1592840e2bd191fafb8c6944b, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.298434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ba37e-a0bd-4f00-ac77-70668a9d0e17 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.298646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d2851-4995-4cf8-80f8-1075f59c9641 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.305005 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c06b1-7680-4e49-b7af-aa9faa7dd9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.305987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c06b1-7680-4e49-b7af-aa9faa7dd9ff admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.306327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c06b1-7680-4e49-b7af-aa9faa7dd9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.306779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c06b1-7680-4e49-b7af-aa9faa7dd9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.307560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c06b1-7680-4e49-b7af-aa9faa7dd9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.309731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c06b1-7680-4e49-b7af-aa9faa7dd9ff admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.319545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 788/1594] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:03 2022] PUT /identity/v3/projects/68ee1f457ed84904b5b4a85a1412ecdc/users/4332f4e1592840e2bd191fafb8c6944b/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:03.319545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ba37e-a0bd-4f00-ac77-70668a9d0e17 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.328735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 807/1595] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:03.328735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c06b1-7680-4e49-b7af-aa9faa7dd9ff admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.342802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c06b1-7680-4e49-b7af-aa9faa7dd9ff admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.426025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1ca15-6b94-4cfe-ae5d-fc88ba3ac558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.427211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1ca15-6b94-4cfe-ae5d-fc88ba3ac558 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.427211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1ca15-6b94-4cfe-ae5d-fc88ba3ac558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.427552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1ca15-6b94-4cfe-ae5d-fc88ba3ac558 admin admin] PATH_INFO: `/v3/projects/7e59a592791b4b5ab8cace0a1a2480cc/users/6a18cf0b93804629a11e0f52f6913d33/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.429034 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ba37e-a0bd-4f00-ac77-70668a9d0e17 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.435029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17d8bc-21c0-448c-b81f-c62f15440a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.436376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17d8bc-21c0-448c-b81f-c62f15440a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.436962 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17d8bc-21c0-448c-b81f-c62f15440a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.437366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17d8bc-21c0-448c-b81f-c62f15440a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.438156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17d8bc-21c0-448c-b81f-c62f15440a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.440098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17d8bc-21c0-448c-b81f-c62f15440a26 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.447721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1ca15-6b94-4cfe-ae5d-fc88ba3ac558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e59a592791b4b5ab8cace0a1a2480cc, user_id=6a18cf0b93804629a11e0f52f6913d33, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.451366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1ca15-6b94-4cfe-ae5d-fc88ba3ac558 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.455968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 789/1596] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:03.459624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17d8bc-21c0-448c-b81f-c62f15440a26 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.471166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17d8bc-21c0-448c-b81f-c62f15440a26 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.478029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 808/1597] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:03 2022] PUT /identity/v3/projects/7e59a592791b4b5ab8cace0a1a2480cc/users/6a18cf0b93804629a11e0f52f6913d33/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:03.478029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1ca15-6b94-4cfe-ae5d-fc88ba3ac558 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.561201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c842df-07b0-4ccb-8185-c61efb2ffdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.562321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c842df-07b0-4ccb-8185-c61efb2ffdda admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.562828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c842df-07b0-4ccb-8185-c61efb2ffdda admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.563364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c842df-07b0-4ccb-8185-c61efb2ffdda admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.564015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1ca15-6b94-4cfe-ae5d-fc88ba3ac558 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.564540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c842df-07b0-4ccb-8185-c61efb2ffdda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.568554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c842df-07b0-4ccb-8185-c61efb2ffdda admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.570259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6231600-9e64-4333-853b-b35ec3b9886b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.571146 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6231600-9e64-4333-853b-b35ec3b9886b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.571512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6231600-9e64-4333-853b-b35ec3b9886b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.571912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6231600-9e64-4333-853b-b35ec3b9886b admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.572731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6231600-9e64-4333-853b-b35ec3b9886b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.574505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6231600-9e64-4333-853b-b35ec3b9886b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.588654 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 809/1598] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:03.589519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6231600-9e64-4333-853b-b35ec3b9886b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.600502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6231600-9e64-4333-853b-b35ec3b9886b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.604456 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 790/1599] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:03 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:03.605777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c842df-07b0-4ccb-8185-c61efb2ffdda admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.609880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bafa5-0c4c-496a-9ac2-f38d1273d9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.610513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bafa5-0c4c-496a-9ac2-f38d1273d9ff admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.610790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bafa5-0c4c-496a-9ac2-f38d1273d9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.611094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bafa5-0c4c-496a-9ac2-f38d1273d9ff admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.611879 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906bafa5-0c4c-496a-9ac2-f38d1273d9ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.613836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906bafa5-0c4c-496a-9ac2-f38d1273d9ff admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.646605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 810/1600] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:03 2022] POST /identity/v3/users => generated 296 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:03.646605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bafa5-0c4c-496a-9ac2-f38d1273d9ff admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.660066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bafa5-0c4c-496a-9ac2-f38d1273d9ff admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.665858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ffae6-4446-4f36-ae20-63184d3a6498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.666725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ffae6-4446-4f36-ae20-63184d3a6498 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.667231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ffae6-4446-4f36-ae20-63184d3a6498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.667624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ffae6-4446-4f36-ae20-63184d3a6498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.668504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ffae6-4446-4f36-ae20-63184d3a6498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.671201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ffae6-4446-4f36-ae20-63184d3a6498 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.689918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c842df-07b0-4ccb-8185-c61efb2ffdda admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.690501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 811/1601] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:03.692520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b829a-632f-4387-b78b-abc9489a6694 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.692953 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b829a-632f-4387-b78b-abc9489a6694 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.693393 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b829a-632f-4387-b78b-abc9489a6694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.696717 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26737127-d77e-411b-a4bb-5c360838098c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.697674 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26737127-d77e-411b-a4bb-5c360838098c service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.698131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26737127-d77e-411b-a4bb-5c360838098c service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.698578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26737127-d77e-411b-a4bb-5c360838098c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.712793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26737127-d77e-411b-a4bb-5c360838098c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.714647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26737127-d77e-411b-a4bb-5c360838098c service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.727956 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 791/1602] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:03.728828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-128b829a-632f-4387-b78b-abc9489a6694 None None] MFA Rules not processed for user `4332f4e1592840e2bd191fafb8c6944b`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:03.729176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26737127-d77e-411b-a4bb-5c360838098c service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.730178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-128b829a-632f-4387-b78b-abc9489a6694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:03.740809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:03.742005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26737127-d77e-411b-a4bb-5c360838098c service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.749346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967a349-df00-4c61-9ac5-b6a006f2a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.750283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967a349-df00-4c61-9ac5-b6a006f2a043 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.750711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967a349-df00-4c61-9ac5-b6a006f2a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.751189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967a349-df00-4c61-9ac5-b6a006f2a043 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.752125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967a349-df00-4c61-9ac5-b6a006f2a043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.754115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967a349-df00-4c61-9ac5-b6a006f2a043 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.763264 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:03.778868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 792/1603] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:03.778868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ab986-dabc-4b02-9bcb-055827ed90f7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.778868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ab986-dabc-4b02-9bcb-055827ed90f7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.778868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ab986-dabc-4b02-9bcb-055827ed90f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.818637 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-aa9ab986-dabc-4b02-9bcb-055827ed90f7 None None] MFA Rules not processed for user `6a18cf0b93804629a11e0f52f6913d33`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:03.819883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-aa9ab986-dabc-4b02-9bcb-055827ed90f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:03.849578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 812/1604] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:03.849578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b829a-632f-4387-b78b-abc9489a6694 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.869489 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b829a-632f-4387-b78b-abc9489a6694 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:03.879779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94b829-42b3-406d-ab15-2be410b37635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:03.880950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94b829-42b3-406d-ab15-2be410b37635 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:03.881451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94b829-42b3-406d-ab15-2be410b37635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:03.881993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94b829-42b3-406d-ab15-2be410b37635 admin admin] PATH_INFO: `/v3/projects/45847b1fd67c4caaa03a36895cd2263a/users/b34e9318826c49ed891dde1f7782898b/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:03.902290 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94b829-42b3-406d-ab15-2be410b37635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45847b1fd67c4caaa03a36895cd2263a, user_id=b34e9318826c49ed891dde1f7782898b, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:03.905958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94b829-42b3-406d-ab15-2be410b37635 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:03.915263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 793/1605] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:03.917411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ab986-dabc-4b02-9bcb-055827ed90f7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.933093 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 813/1606] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:03 2022] PUT /identity/v3/projects/45847b1fd67c4caaa03a36895cd2263a/users/b34e9318826c49ed891dde1f7782898b/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:03.936941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94b829-42b3-406d-ab15-2be410b37635 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:03.946042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ab986-dabc-4b02-9bcb-055827ed90f7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:04.028052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94b829-42b3-406d-ab15-2be410b37635 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:04.034803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58689f6-a6ad-4ad5-aea7-f2f4b81edcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:04.035502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58689f6-a6ad-4ad5-aea7-f2f4b81edcd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.036422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58689f6-a6ad-4ad5-aea7-f2f4b81edcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.036716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58689f6-a6ad-4ad5-aea7-f2f4b81edcd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.037395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58689f6-a6ad-4ad5-aea7-f2f4b81edcd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:04.039137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58689f6-a6ad-4ad5-aea7-f2f4b81edcd6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:04.056684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 814/1607] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:04.056684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58689f6-a6ad-4ad5-aea7-f2f4b81edcd6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:04.069132 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58689f6-a6ad-4ad5-aea7-f2f4b81edcd6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:04.074004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370499d-ae4e-4373-913b-4b8d7dd5fc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:04.074684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370499d-ae4e-4373-913b-4b8d7dd5fc87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.075173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370499d-ae4e-4373-913b-4b8d7dd5fc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.075459 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370499d-ae4e-4373-913b-4b8d7dd5fc87 admin admin] PATH_INFO: `/v3/projects/329d492f457d4bc6bf63ff3eb5971826/users/a716e14c35014a728f0dc76675ff6874/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.079064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dff13-6fc3-4db3-9caf-1857a5edd9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:04.080392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dff13-6fc3-4db3-9caf-1857a5edd9fb admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.080891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dff13-6fc3-4db3-9caf-1857a5edd9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.081522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dff13-6fc3-4db3-9caf-1857a5edd9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.083130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dff13-6fc3-4db3-9caf-1857a5edd9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:04.086681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dff13-6fc3-4db3-9caf-1857a5edd9fb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:04.096953 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370499d-ae4e-4373-913b-4b8d7dd5fc87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=329d492f457d4bc6bf63ff3eb5971826, user_id=a716e14c35014a728f0dc76675ff6874, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:04.101879 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370499d-ae4e-4373-913b-4b8d7dd5fc87 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:04.105273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 794/1608] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:04.106238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dff13-6fc3-4db3-9caf-1857a5edd9fb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:04.122312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dff13-6fc3-4db3-9caf-1857a5edd9fb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:04.138215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 815/1609] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:04 2022] PUT /identity/v3/projects/329d492f457d4bc6bf63ff3eb5971826/users/a716e14c35014a728f0dc76675ff6874/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:04.140733 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370499d-ae4e-4373-913b-4b8d7dd5fc87 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:04.199059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619543ae-b382-419a-9efc-fcfa57eb6cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:04.199821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619543ae-b382-419a-9efc-fcfa57eb6cad admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.200178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619543ae-b382-419a-9efc-fcfa57eb6cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.200553 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619543ae-b382-419a-9efc-fcfa57eb6cad admin admin] PATH_INFO: `/v3/projects/2abeb0b54520461cb414ea9c0599380f/users/624f8fd9a74e48afa2f8b74b090a8dd6/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.223503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619543ae-b382-419a-9efc-fcfa57eb6cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abeb0b54520461cb414ea9c0599380f, user_id=624f8fd9a74e48afa2f8b74b090a8dd6, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:04.226678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619543ae-b382-419a-9efc-fcfa57eb6cad admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:04.252133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 795/1610] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:04 2022] PUT /identity/v3/projects/2abeb0b54520461cb414ea9c0599380f/users/624f8fd9a74e48afa2f8b74b090a8dd6/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:04.252133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f485e-247e-4ddf-bec7-e21c17da213c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.252133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f485e-247e-4ddf-bec7-e21c17da213c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.252133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f485e-247e-4ddf-bec7-e21c17da213c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.274358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2f2f485e-247e-4ddf-bec7-e21c17da213c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:04.275873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2f2f485e-247e-4ddf-bec7-e21c17da213c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:04.281891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:04.308636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370499d-ae4e-4373-913b-4b8d7dd5fc87 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:04.320021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:04.366801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 796/1611] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:04.366801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd0aff-a89f-4004-9975-caddccd3475d None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.366801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd0aff-a89f-4004-9975-caddccd3475d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.366801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd0aff-a89f-4004-9975-caddccd3475d None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.373323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 797/1612] 10.222.0.16 () {54 vars in 961 bytes} [Thu Sep 22 17:07:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:07:04.375611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd0aff-a89f-4004-9975-caddccd3475d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:04.401937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd0aff-a89f-4004-9975-caddccd3475d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:04.406828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a202221-7f32-49aa-b19d-fc9bb941c140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:04.407672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a202221-7f32-49aa-b19d-fc9bb941c140 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.407956 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a202221-7f32-49aa-b19d-fc9bb941c140 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.408280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a202221-7f32-49aa-b19d-fc9bb941c140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.415873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f51ed-3f7c-424b-b927-b1495d24ad48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:04.416605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f51ed-3f7c-424b-b927-b1495d24ad48 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.417179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f51ed-3f7c-424b-b927-b1495d24ad48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.417601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f51ed-3f7c-424b-b927-b1495d24ad48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.419296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f51ed-3f7c-424b-b927-b1495d24ad48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:04.423201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f51ed-3f7c-424b-b927-b1495d24ad48 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:04.446764 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 798/1613] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:04.448635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f51ed-3f7c-424b-b927-b1495d24ad48 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:04.468761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f51ed-3f7c-424b-b927-b1495d24ad48 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:04.477095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca81d5-467a-4503-882e-aa6478a55ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:04.478161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca81d5-467a-4503-882e-aa6478a55ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.478660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca81d5-467a-4503-882e-aa6478a55ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.479192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca81d5-467a-4503-882e-aa6478a55ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.480964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca81d5-467a-4503-882e-aa6478a55ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:04.484148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca81d5-467a-4503-882e-aa6478a55ccc admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:04.495868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a202221-7f32-49aa-b19d-fc9bb941c140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:04.497529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a202221-7f32-49aa-b19d-fc9bb941c140 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:04.506443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 799/1614] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:04.508936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e48eea-cb09-49a4-8649-700d02f4b83b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.510726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e48eea-cb09-49a4-8649-700d02f4b83b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.511356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e48eea-cb09-49a4-8649-700d02f4b83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.513228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 816/1615] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:04.516140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a4a58-4044-4d01-bd39-58ea2989d63a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.516629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a4a58-4044-4d01-bd39-58ea2989d63a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.516629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a4a58-4044-4d01-bd39-58ea2989d63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.538403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-4a5a4a58-4044-4d01-bd39-58ea2989d63a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:04.539677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4a5a4a58-4044-4d01-bd39-58ea2989d63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:04.554445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 817/1616] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:04.556369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a4a58-4044-4d01-bd39-58ea2989d63a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:04.580881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-53e48eea-cb09-49a4-8649-700d02f4b83b None None] MFA Rules not processed for user `a716e14c35014a728f0dc76675ff6874`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:04.582350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-53e48eea-cb09-49a4-8649-700d02f4b83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:04.668349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a4a58-4044-4d01-bd39-58ea2989d63a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:04.672977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce56e7-1f89-43b1-aa23-9602e7278eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:04.673673 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce56e7-1f89-43b1-aa23-9602e7278eee service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.674081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce56e7-1f89-43b1-aa23-9602e7278eee service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.674267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce56e7-1f89-43b1-aa23-9602e7278eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.689820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce56e7-1f89-43b1-aa23-9602e7278eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:04.691438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce56e7-1f89-43b1-aa23-9602e7278eee service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:04.704127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 818/1617] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:04.705635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbe70e-802f-4d2b-857c-648647dc15f8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.705954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbe70e-802f-4d2b-857c-648647dc15f8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.706294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbe70e-802f-4d2b-857c-648647dc15f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.709677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 800/1618] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:04.710677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e48eea-cb09-49a4-8649-700d02f4b83b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:04.727809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e48eea-cb09-49a4-8649-700d02f4b83b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:04.733480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135d1fd-4380-4a02-a46e-6241528defc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:04.734305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d1fd-4380-4a02-a46e-6241528defc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.734584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d1fd-4380-4a02-a46e-6241528defc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.734862 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d1fd-4380-4a02-a46e-6241528defc9 admin admin] PATH_INFO: `/v3/projects/105c11557b6f45329dc97af01e8ea915/users/bc8c2a5413c943ed94d76eceb2720f17/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.757667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-12cbe70e-802f-4d2b-857c-648647dc15f8 None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:04.759299 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-12cbe70e-802f-4d2b-857c-648647dc15f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:04.788714 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135d1fd-4380-4a02-a46e-6241528defc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=105c11557b6f45329dc97af01e8ea915, user_id=bc8c2a5413c943ed94d76eceb2720f17, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:04.791596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135d1fd-4380-4a02-a46e-6241528defc9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:04.831199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 801/1619] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:04 2022] PUT /identity/v3/projects/105c11557b6f45329dc97af01e8ea915/users/bc8c2a5413c943ed94d76eceb2720f17/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:04.832366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135d1fd-4380-4a02-a46e-6241528defc9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:04.852350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:04.922274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 819/1620] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Sep 22 17:07:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:04.930351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:04.936252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbe70e-802f-4d2b-857c-648647dc15f8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:04.949183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135d1fd-4380-4a02-a46e-6241528defc9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:04.958186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb21c68-5c94-4bd6-bb60-38323de87e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:04.959473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb21c68-5c94-4bd6-bb60-38323de87e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:04.959838 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb21c68-5c94-4bd6-bb60-38323de87e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:04.960207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb21c68-5c94-4bd6-bb60-38323de87e9f admin admin] PATH_INFO: `/v3/projects/a6e3ace80a4a43b2b187060d6eca8538/users/5feea3b3b7e4401ea7db242cb60fdfcc/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:04.981444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbe70e-802f-4d2b-857c-648647dc15f8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:04.986558 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb21c68-5c94-4bd6-bb60-38323de87e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e3ace80a4a43b2b187060d6eca8538, user_id=5feea3b3b7e4401ea7db242cb60fdfcc, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:04.992129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb21c68-5c94-4bd6-bb60-38323de87e9f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:05.036209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 802/1621] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:04 2022] PUT /identity/v3/projects/a6e3ace80a4a43b2b187060d6eca8538/users/5feea3b3b7e4401ea7db242cb60fdfcc/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:05.038735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb21c68-5c94-4bd6-bb60-38323de87e9f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:05.139557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1ada3-be5a-43dd-95f0-37ed87e2f2f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:05.140206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1ada3-be5a-43dd-95f0-37ed87e2f2f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:05.140485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1ada3-be5a-43dd-95f0-37ed87e2f2f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:05.140762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1ada3-be5a-43dd-95f0-37ed87e2f2f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:05.162579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1ada3-be5a-43dd-95f0-37ed87e2f2f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:05.164316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1ada3-be5a-43dd-95f0-37ed87e2f2f1 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:05.184365 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 820/1622] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:05.187242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f0c23-278e-432c-8161-57d9d22b0f15 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:05.187790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f0c23-278e-432c-8161-57d9d22b0f15 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:05.188286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f0c23-278e-432c-8161-57d9d22b0f15 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:05.190381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 821/1623] 10.222.0.16 () {54 vars in 961 bytes} [Thu Sep 22 17:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:07:05.195280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db76dc7-8ebf-44e7-9d83-119af0fbe913 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:05.195869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db76dc7-8ebf-44e7-9d83-119af0fbe913 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:05.197244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db76dc7-8ebf-44e7-9d83-119af0fbe913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:05.229644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6db76dc7-8ebf-44e7-9d83-119af0fbe913 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:05.230916 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6db76dc7-8ebf-44e7-9d83-119af0fbe913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:05.247284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb21c68-5c94-4bd6-bb60-38323de87e9f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:05.260995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e04d42-c530-40d9-b879-d23239d827bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:05.261615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e04d42-c530-40d9-b879-d23239d827bb service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:05.261896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e04d42-c530-40d9-b879-d23239d827bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:05.262167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e04d42-c530-40d9-b879-d23239d827bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:05.386637 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 822/1624] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:05.388248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db76dc7-8ebf-44e7-9d83-119af0fbe913 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:05.411598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e04d42-c530-40d9-b879-d23239d827bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:05.416386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e04d42-c530-40d9-b879-d23239d827bb service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:05.439657 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 803/1625] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:05.456004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd559451-3cae-4023-aaed-dd41645be388 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:05.463365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd559451-3cae-4023-aaed-dd41645be388 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:05.466147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd559451-3cae-4023-aaed-dd41645be388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:05.524154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-dd559451-3cae-4023-aaed-dd41645be388 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:05.525592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-dd559451-3cae-4023-aaed-dd41645be388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:05.603965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db76dc7-8ebf-44e7-9d83-119af0fbe913 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:05.605709 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 804/1626] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:05.608659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f855cca-7b7f-487e-91d3-f9164ec5c117 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:05.609810 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f855cca-7b7f-487e-91d3-f9164ec5c117 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:05.610627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f855cca-7b7f-487e-91d3-f9164ec5c117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:05.631574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b40430-a5cd-4234-945b-afc9de2faa16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:05.632214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b40430-a5cd-4234-945b-afc9de2faa16 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:05.632500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b40430-a5cd-4234-945b-afc9de2faa16 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:05.632777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b40430-a5cd-4234-945b-afc9de2faa16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:05.657965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:05.691092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6f855cca-7b7f-487e-91d3-f9164ec5c117 None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:05.693263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6f855cca-7b7f-487e-91d3-f9164ec5c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:05.723172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 805/1627] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Sep 22 17:07:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:05.724189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f855cca-7b7f-487e-91d3-f9164ec5c117 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:05.804789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b40430-a5cd-4234-945b-afc9de2faa16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:05.806915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b40430-a5cd-4234-945b-afc9de2faa16 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:05.848605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 823/1628] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:05.849801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b40430-a5cd-4234-945b-afc9de2faa16 service nova] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:05.884005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f855cca-7b7f-487e-91d3-f9164ec5c117 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:05.894292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba18a05-0581-4867-9029-54ba9464a73a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:05.895343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b40430-a5cd-4234-945b-afc9de2faa16 service nova] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:05.895545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba18a05-0581-4867-9029-54ba9464a73a service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:05.896481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba18a05-0581-4867-9029-54ba9464a73a service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:05.896769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba18a05-0581-4867-9029-54ba9464a73a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:05.913911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b290391-be98-4afb-8df2-65dbd8c417ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:05.914487 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b290391-be98-4afb-8df2-65dbd8c417ed service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:05.914771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b290391-be98-4afb-8df2-65dbd8c417ed service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:05.915110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b290391-be98-4afb-8df2-65dbd8c417ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:05.918860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba18a05-0581-4867-9029-54ba9464a73a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:05.920023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba18a05-0581-4867-9029-54ba9464a73a service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:05.947031 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 806/1629] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:05.948370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba18a05-0581-4867-9029-54ba9464a73a service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:05.968014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba18a05-0581-4867-9029-54ba9464a73a service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:05.978969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db176b5-3050-4d00-af83-0ce72fa18d3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:05.980000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db176b5-3050-4d00-af83-0ce72fa18d3a service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:05.980519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db176b5-3050-4d00-af83-0ce72fa18d3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:05.981000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db176b5-3050-4d00-af83-0ce72fa18d3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:06.020513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db176b5-3050-4d00-af83-0ce72fa18d3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:06.025451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db176b5-3050-4d00-af83-0ce72fa18d3a service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:06.052316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 807/1630] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:06.064346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b290391-be98-4afb-8df2-65dbd8c417ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:06.065995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b290391-be98-4afb-8df2-65dbd8c417ed service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:06.087121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 824/1631] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:06.368515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129b4d7-72f9-4da0-8baf-1bc81d2cfc16 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:06.369218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129b4d7-72f9-4da0-8baf-1bc81d2cfc16 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:06.369842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129b4d7-72f9-4da0-8baf-1bc81d2cfc16 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:06.371715 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 808/1632] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 22 17:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:07:06.380069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e683-4d19-4726-bfc3-c4fe327b4999 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:06.380762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e683-4d19-4726-bfc3-c4fe327b4999 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:06.382279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e683-4d19-4726-bfc3-c4fe327b4999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:06.433434 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6030e683-4d19-4726-bfc3-c4fe327b4999 None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:06.435236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6030e683-4d19-4726-bfc3-c4fe327b4999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:06.485846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 825/1633] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:07:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:06.491907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129b4d7-72f9-4da0-8baf-1bc81d2cfc16 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:06.517617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129b4d7-72f9-4da0-8baf-1bc81d2cfc16 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:06.525022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8732c-e7bb-4917-b033-639c4c627d3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:06.529265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8732c-e7bb-4917-b033-639c4c627d3c service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:06.531518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8732c-e7bb-4917-b033-639c4c627d3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:06.532961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8732c-e7bb-4917-b033-639c4c627d3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:06.559234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:06.574894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8732c-e7bb-4917-b033-639c4c627d3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:06.591401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8732c-e7bb-4917-b033-639c4c627d3c service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:06.656448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 809/1634] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:07:06.664598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a55d8c-fcd8-4c28-9810-a596195af557 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:06.665293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a55d8c-fcd8-4c28-9810-a596195af557 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:06.665466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a55d8c-fcd8-4c28-9810-a596195af557 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:06.666867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 826/1635] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 22 17:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:07:06.674836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb17f9-3d10-496a-b785-49b215ccd58d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:06.675337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb17f9-3d10-496a-b785-49b215ccd58d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:06.676065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb17f9-3d10-496a-b785-49b215ccd58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:06.723519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f7eb17f9-3d10-496a-b785-49b215ccd58d None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:06.725499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f7eb17f9-3d10-496a-b785-49b215ccd58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:06.751580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 810/1636] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:07:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:06.805643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a55d8c-fcd8-4c28-9810-a596195af557 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:06.833461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a55d8c-fcd8-4c28-9810-a596195af557 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:06.843757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cd2a9-8236-40fe-b5e7-1696315412d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:06.845863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd2a9-8236-40fe-b5e7-1696315412d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:06.846925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd2a9-8236-40fe-b5e7-1696315412d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:06.848092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd2a9-8236-40fe-b5e7-1696315412d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:06.990574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cd2a9-8236-40fe-b5e7-1696315412d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:06.990574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cd2a9-8236-40fe-b5e7-1696315412d5 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:06.990574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 827/1637] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:07:10.797568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb17f9-3d10-496a-b785-49b215ccd58d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:10.815000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb17f9-3d10-496a-b785-49b215ccd58d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:10.842774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:10.847871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:10.849112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:10.849786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] PATH_INFO: `/v3/users/e94051ebd36148e890cb9e75d98eb579` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:10.853716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e94051ebd36148e890cb9e75d98eb579)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:10.857402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:07:10.857402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 22 17:07:10.857402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: Sep 22 17:07:10.863518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:11.123890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e94051ebd36148e890cb9e75d98eb579'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:11.150629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e94051ebd36148e890cb9e75d98eb579'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:11.174504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e94051ebd36148e890cb9e75d98eb579'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:11.175273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e94051ebd36148e890cb9e75d98eb579'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:11.191522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e94051ebd36148e890cb9e75d98eb579'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:11.205688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 811/1638] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:10 2022] DELETE /identity/v3/users/e94051ebd36148e890cb9e75d98eb579 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:11.207492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8988a9-5cd2-4e5b-97bf-2d74200baee0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:11.208022 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8988a9-5cd2-4e5b-97bf-2d74200baee0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:11.208022 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8988a9-5cd2-4e5b-97bf-2d74200baee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:11.256593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7c8988a9-5cd2-4e5b-97bf-2d74200baee0 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:11.260157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7c8988a9-5cd2-4e5b-97bf-2d74200baee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:11.416881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 828/1639] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:11.585185 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:11.587272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:11.738253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b12c60-ac78-4cc4-8e15-06d219bde616 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:11.742373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba622d-766c-4a30-bbd0-7d164ce646a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:11.743078 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba622d-766c-4a30-bbd0-7d164ce646a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:11.743381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba622d-766c-4a30-bbd0-7d164ce646a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:11.743856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba622d-766c-4a30-bbd0-7d164ce646a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:11.761925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba622d-766c-4a30-bbd0-7d164ce646a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:11.761925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba622d-766c-4a30-bbd0-7d164ce646a3 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:11.784152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8988a9-5cd2-4e5b-97bf-2d74200baee0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:11.784152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7c8988a9-5cd2-4e5b-97bf-2d74200baee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:11.801002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 812/1640] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:07:11.801741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8988a9-5cd2-4e5b-97bf-2d74200baee0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:11.815528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:11.816143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:11.816546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:11.816941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] PATH_INFO: `/v3/users/6a18cf0b93804629a11e0f52f6913d33` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:11.822033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a18cf0b93804629a11e0f52f6913d33)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:11.824261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:11.938117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:11.993437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a18cf0b93804629a11e0f52f6913d33'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:12.031491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a18cf0b93804629a11e0f52f6913d33'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:12.053183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a18cf0b93804629a11e0f52f6913d33'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:12.082804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a18cf0b93804629a11e0f52f6913d33'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:12.084537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a18cf0b93804629a11e0f52f6913d33'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:12.109289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 829/1641] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:11 2022] DELETE /identity/v3/users/6a18cf0b93804629a11e0f52f6913d33 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:12.110450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a399d3-7ce9-444e-bb1a-a33d21cb53da None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:12.110785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a399d3-7ce9-444e-bb1a-a33d21cb53da None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:12.111258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a399d3-7ce9-444e-bb1a-a33d21cb53da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:12.149948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c6a399d3-7ce9-444e-bb1a-a33d21cb53da None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:12.151063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c6a399d3-7ce9-444e-bb1a-a33d21cb53da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:12.287939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 813/1642] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:12.393786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:12.399226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:12.605094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e8c99-e4e8-4df2-b8cd-8031a4f9aaa5 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:12.647616 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510a94a-2d51-4dfc-bfd4-ac679eea2fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:12.648465 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510a94a-2d51-4dfc-bfd4-ac679eea2fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:12.648465 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510a94a-2d51-4dfc-bfd4-ac679eea2fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:12.648723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510a94a-2d51-4dfc-bfd4-ac679eea2fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:12.673539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510a94a-2d51-4dfc-bfd4-ac679eea2fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:12.677830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510a94a-2d51-4dfc-bfd4-ac679eea2fd6 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:12.690683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 830/1643] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:13.231904 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a399d3-7ce9-444e-bb1a-a33d21cb53da None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:13.236787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c6a399d3-7ce9-444e-bb1a-a33d21cb53da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:13.250508 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:13.272641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a399d3-7ce9-444e-bb1a-a33d21cb53da None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:13.296302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:13.297585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:13.298080 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:13.298585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] PATH_INFO: `/v3/projects/d992b97fcee74a69baacab0b30f11d57` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:13.301784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d992b97fcee74a69baacab0b30f11d57)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:13.304997 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:13.428372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd992b97fcee74a69baacab0b30f11d57', 'request_id': 'req-e71ce6f1-8660-4491-ae08-042fbc475772'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:13.442687 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd992b97fcee74a69baacab0b30f11d57', 'request_id': 'req-e71ce6f1-8660-4491-ae08-042fbc475772'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:13.468590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd992b97fcee74a69baacab0b30f11d57', 'request_id': 'req-e71ce6f1-8660-4491-ae08-042fbc475772'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:13.486267 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] The token cache is being invalidate because project d992b97fcee74a69baacab0b30f11d57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:13.486738 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:13.487764 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:13.490347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 814/1644] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:13 2022] DELETE /identity/v3/projects/d992b97fcee74a69baacab0b30f11d57 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:13.494346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510a94a-2d51-4dfc-bfd4-ac679eea2fd6 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:13.505695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510a94a-2d51-4dfc-bfd4-ac679eea2fd6 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:13.631439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef511a0-a507-4861-a751-93b28e7681ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:13.632558 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef511a0-a507-4861-a751-93b28e7681ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:13.633066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef511a0-a507-4861-a751-93b28e7681ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:13.633555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef511a0-a507-4861-a751-93b28e7681ec admin admin] PATH_INFO: `/v3/users/4332f4e1592840e2bd191fafb8c6944b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:13.635562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef511a0-a507-4861-a751-93b28e7681ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=4332f4e1592840e2bd191fafb8c6944b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:13.643748 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef511a0-a507-4861-a751-93b28e7681ec admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:13.814940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9ef511a0-a507-4861-a751-93b28e7681ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4332f4e1592840e2bd191fafb8c6944b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:13.819627 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:13.908860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9ef511a0-a507-4861-a751-93b28e7681ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4332f4e1592840e2bd191fafb8c6944b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:13.937692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9ef511a0-a507-4861-a751-93b28e7681ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4332f4e1592840e2bd191fafb8c6944b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:13.970299 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9ef511a0-a507-4861-a751-93b28e7681ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4332f4e1592840e2bd191fafb8c6944b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:13.972197 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9ef511a0-a507-4861-a751-93b28e7681ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4332f4e1592840e2bd191fafb8c6944b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:14.001895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 831/1645] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:13 2022] DELETE /identity/v3/users/4332f4e1592840e2bd191fafb8c6944b => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:14.674821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:14.676844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:14.707107 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ce6f1-8660-4491-ae08-042fbc475772 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:14.869714 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:14.870647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:14.871735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:14.872019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] PATH_INFO: `/v3/projects/7e59a592791b4b5ab8cace0a1a2480cc` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:14.881983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e59a592791b4b5ab8cace0a1a2480cc)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:14.884096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:15.040803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e59a592791b4b5ab8cace0a1a2480cc', 'request_id': 'req-d25046ff-b9a0-4519-a5b2-21b1326e437d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:15.059807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e59a592791b4b5ab8cace0a1a2480cc', 'request_id': 'req-d25046ff-b9a0-4519-a5b2-21b1326e437d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:15.072625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e59a592791b4b5ab8cace0a1a2480cc', 'request_id': 'req-d25046ff-b9a0-4519-a5b2-21b1326e437d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:15.119161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] The token cache is being invalidate because project 7e59a592791b4b5ab8cace0a1a2480cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:15.119825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:15.122589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d25046ff-b9a0-4519-a5b2-21b1326e437d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:15.128406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 815/1646] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:14 2022] DELETE /identity/v3/projects/7e59a592791b4b5ab8cace0a1a2480cc => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:15.135483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c71cde-f572-4439-b482-70df8be269b1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:15.140721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c71cde-f572-4439-b482-70df8be269b1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:15.141246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c71cde-f572-4439-b482-70df8be269b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:15.193533 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f2c71cde-f572-4439-b482-70df8be269b1 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:15.198316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f2c71cde-f572-4439-b482-70df8be269b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:15.386735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 832/1647] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:15.387934 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd7ed5-42fd-4d10-ae6d-17aa5faf005b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:15.388324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd7ed5-42fd-4d10-ae6d-17aa5faf005b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:15.388749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd7ed5-42fd-4d10-ae6d-17aa5faf005b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:15.448103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-81bd7ed5-42fd-4d10-ae6d-17aa5faf005b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:15.455818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-81bd7ed5-42fd-4d10-ae6d-17aa5faf005b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:15.485454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 816/1648] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:15.501698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c71cde-f572-4439-b482-70df8be269b1 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:15.502068 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be726b3d-0713-4ac0-be83-a6d6cdcbd0e0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:15.502436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be726b3d-0713-4ac0-be83-a6d6cdcbd0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:15.502788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be726b3d-0713-4ac0-be83-a6d6cdcbd0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:15.528624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c71cde-f572-4439-b482-70df8be269b1 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:15.546325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19459ec9-2307-4caf-b7d7-a77362c2a91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:15.546971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19459ec9-2307-4caf-b7d7-a77362c2a91b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:15.547297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19459ec9-2307-4caf-b7d7-a77362c2a91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:15.547573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19459ec9-2307-4caf-b7d7-a77362c2a91b admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:15.548244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19459ec9-2307-4caf-b7d7-a77362c2a91b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:15.548542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19459ec9-2307-4caf-b7d7-a77362c2a91b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:15.550629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19459ec9-2307-4caf-b7d7-a77362c2a91b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:15.558453 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:15.578760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-be726b3d-0713-4ac0-be83-a6d6cdcbd0e0 None None] MFA Rules not processed for user `3725446c6d5040c7870b056d4bfbeca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:15.580994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-be726b3d-0713-4ac0-be83-a6d6cdcbd0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:15.583651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 833/1649] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:15.588324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19459ec9-2307-4caf-b7d7-a77362c2a91b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:15.610182 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19459ec9-2307-4caf-b7d7-a77362c2a91b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:15.625394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db4ec7-9dff-4b6b-bbf6-e1de9f81cc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:15.626131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4ec7-9dff-4b6b-bbf6-e1de9f81cc59 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:15.627261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4ec7-9dff-4b6b-bbf6-e1de9f81cc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:15.627587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4ec7-9dff-4b6b-bbf6-e1de9f81cc59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:15.628494 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db4ec7-9dff-4b6b-bbf6-e1de9f81cc59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:15.639843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db4ec7-9dff-4b6b-bbf6-e1de9f81cc59 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:15.702316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 834/1650] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:15 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:15.706834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db4ec7-9dff-4b6b-bbf6-e1de9f81cc59 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:15.770664 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 817/1651] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:15.782658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd8dc8-1238-400f-b312-dede931c3f53 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:15.782908 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd8dc8-1238-400f-b312-dede931c3f53 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:15.783409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd8dc8-1238-400f-b312-dede931c3f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:15.824076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5cfd8dc8-1238-400f-b312-dede931c3f53 None None] MFA Rules not processed for user `3725446c6d5040c7870b056d4bfbeca6`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:15.825669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5cfd8dc8-1238-400f-b312-dede931c3f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:15.867656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db4ec7-9dff-4b6b-bbf6-e1de9f81cc59 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:15.886837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d1955-72a1-43ac-b07d-b737d299688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:15.887507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d1955-72a1-43ac-b07d-b737d299688e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:15.887995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d1955-72a1-43ac-b07d-b737d299688e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:15.888092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d1955-72a1-43ac-b07d-b737d299688e admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:15.888762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d1955-72a1-43ac-b07d-b737d299688e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:15.892704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d1955-72a1-43ac-b07d-b737d299688e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:15.945834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 835/1652] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:15 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:15.948146 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d1955-72a1-43ac-b07d-b737d299688e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:15.953727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c23d1955-72a1-43ac-b07d-b737d299688e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:15.987929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 818/1653] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:15.991191 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d1955-72a1-43ac-b07d-b737d299688e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:15.993681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd8dc8-1238-400f-b312-dede931c3f53 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:16.006281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd8dc8-1238-400f-b312-dede931c3f53 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:16.007547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:16.008494 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:16.009200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:16.009487 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] PATH_INFO: `/v3/projects/68ee1f457ed84904b5b4a85a1412ecdc` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:16.011909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68ee1f457ed84904b5b4a85a1412ecdc)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:16.013139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 17:07:16.013139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Sep 22 17:07:16.013139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: Sep 22 17:07:16.013891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0c28c-11d6-4aa9-a5f0-40cc9dad208a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:16.014468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:16.014813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0c28c-11d6-4aa9-a5f0-40cc9dad208a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:16.015216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0c28c-11d6-4aa9-a5f0-40cc9dad208a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:16.015578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0c28c-11d6-4aa9-a5f0-40cc9dad208a admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:16.017025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0c28c-11d6-4aa9-a5f0-40cc9dad208a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:16.018811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0c28c-11d6-4aa9-a5f0-40cc9dad208a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:16.045725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 819/1654] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:16.049646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0c28c-11d6-4aa9-a5f0-40cc9dad208a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:16.052454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a9c0c28c-11d6-4aa9-a5f0-40cc9dad208a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:16.080711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:16.184342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ee1f457ed84904b5b4a85a1412ecdc', 'request_id': 'req-925f584d-af35-4f41-aa76-5b360335e4e2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:16.223531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ee1f457ed84904b5b4a85a1412ecdc', 'request_id': 'req-925f584d-af35-4f41-aa76-5b360335e4e2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:16.246202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ee1f457ed84904b5b4a85a1412ecdc', 'request_id': 'req-925f584d-af35-4f41-aa76-5b360335e4e2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:16.269790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0c28c-11d6-4aa9-a5f0-40cc9dad208a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:16.272884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] The token cache is being invalidate because project 68ee1f457ed84904b5b4a85a1412ecdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:16.273341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:16.276247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:16.281340 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 836/1655] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:15 2022] DELETE /identity/v3/projects/68ee1f457ed84904b5b4a85a1412ecdc => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:16.284502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:16.288367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:16.389946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:16.474993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f584d-af35-4f41-aa76-5b360335e4e2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:16.485797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2693872-ac42-4ddb-91da-411d7f804741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:16.486897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2693872-ac42-4ddb-91da-411d7f804741 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:16.487423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2693872-ac42-4ddb-91da-411d7f804741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:16.487916 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2693872-ac42-4ddb-91da-411d7f804741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:16.488983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2693872-ac42-4ddb-91da-411d7f804741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:16.493813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db29799-6c28-4596-a8fa-88bb36820c4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:16.494450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db29799-6c28-4596-a8fa-88bb36820c4e service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:16.494727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db29799-6c28-4596-a8fa-88bb36820c4e service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:16.495020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db29799-6c28-4596-a8fa-88bb36820c4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:16.496480 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2693872-ac42-4ddb-91da-411d7f804741 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:16.499518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6db29799-6c28-4596-a8fa-88bb36820c4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:16.537008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 837/1656] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:16.538562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd00ae-615a-41b8-9c04-96c5429a701e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:16.538839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd00ae-615a-41b8-9c04-96c5429a701e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:16.539157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd00ae-615a-41b8-9c04-96c5429a701e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:16.609992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-29dd00ae-615a-41b8-9c04-96c5429a701e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:16.616987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-29dd00ae-615a-41b8-9c04-96c5429a701e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:16.656125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 838/1657] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:16.656306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db29799-6c28-4596-a8fa-88bb36820c4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:16.659144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db29799-6c28-4596-a8fa-88bb36820c4e service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:16.665088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd00ae-615a-41b8-9c04-96c5429a701e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:16.673719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 820/1658] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:16.676678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6f759-a382-4618-a6f3-bb8e21267f13 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:16.677015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6f759-a382-4618-a6f3-bb8e21267f13 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:16.677924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6f759-a382-4618-a6f3-bb8e21267f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:16.689725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd00ae-615a-41b8-9c04-96c5429a701e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:16.710789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436b336-659c-4069-9ea1-f454fef20236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:16.713272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436b336-659c-4069-9ea1-f454fef20236 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:16.713595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436b336-659c-4069-9ea1-f454fef20236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:16.713876 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436b336-659c-4069-9ea1-f454fef20236 admin admin] PATH_INFO: `/v3/projects/9c704cc59df6456ea25ca05356b23384/users/e5faa2cd2f4c4c52902d1c21f3388377/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:16.740168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-19f6f759-a382-4618-a6f3-bb8e21267f13 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:16.741410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-19f6f759-a382-4618-a6f3-bb8e21267f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:16.772964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436b336-659c-4069-9ea1-f454fef20236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c704cc59df6456ea25ca05356b23384, user_id=e5faa2cd2f4c4c52902d1c21f3388377, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:16.776784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 821/1659] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:16.783331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6f759-a382-4618-a6f3-bb8e21267f13 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:16.785186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436b336-659c-4069-9ea1-f454fef20236 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:16.842633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6f759-a382-4618-a6f3-bb8e21267f13 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:16.867008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 839/1660] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:16 2022] PUT /identity/v3/projects/9c704cc59df6456ea25ca05356b23384/users/e5faa2cd2f4c4c52902d1c21f3388377/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:16.873465 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e895c-2db6-494d-aa78-068856b9c25b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:16.873885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e895c-2db6-494d-aa78-068856b9c25b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:16.874168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e895c-2db6-494d-aa78-068856b9c25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:16.992368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-481e895c-2db6-494d-aa78-068856b9c25b None None] MFA Rules not processed for user `e5faa2cd2f4c4c52902d1c21f3388377`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:16.993648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-481e895c-2db6-494d-aa78-068856b9c25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:17.149258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd24a8-e84b-45ef-884a-673f2e83d5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:17.150986 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd24a8-e84b-45ef-884a-673f2e83d5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:17.151490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd24a8-e84b-45ef-884a-673f2e83d5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:17.152054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd24a8-e84b-45ef-884a-673f2e83d5c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:17.154846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd24a8-e84b-45ef-884a-673f2e83d5c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:17.157922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd24a8-e84b-45ef-884a-673f2e83d5c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:17.162434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd24a8-e84b-45ef-884a-673f2e83d5c5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:17.175294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:17.180476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 840/1661] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:17.195019 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e895c-2db6-494d-aa78-068856b9c25b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:17.200038 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-481e895c-2db6-494d-aa78-068856b9c25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:17.217093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 822/1662] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:17.218557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd24a8-e84b-45ef-884a-673f2e83d5c5 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:17.245152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd24a8-e84b-45ef-884a-673f2e83d5c5 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:17.289863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b80af-313f-43db-a285-94aa533c1902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:17.291122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b80af-313f-43db-a285-94aa533c1902 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:17.292054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b80af-313f-43db-a285-94aa533c1902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:17.292372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b80af-313f-43db-a285-94aa533c1902 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:17.294002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b80af-313f-43db-a285-94aa533c1902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:17.302437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b80af-313f-43db-a285-94aa533c1902 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:17.385597 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e895c-2db6-494d-aa78-068856b9c25b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:17.400140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db8b54-5fe1-4352-accd-e6dbf9081de4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:17.403247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db8b54-5fe1-4352-accd-e6dbf9081de4 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:17.403527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db8b54-5fe1-4352-accd-e6dbf9081de4 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:17.403941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db8b54-5fe1-4352-accd-e6dbf9081de4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:17.408497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 823/1663] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:17 2022] POST /identity/v3/projects => generated 369 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:17.410618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b80af-313f-43db-a285-94aa533c1902 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:17.426551 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db8b54-5fe1-4352-accd-e6dbf9081de4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:17.429411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db8b54-5fe1-4352-accd-e6dbf9081de4 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:17.470987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 841/1664] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:17.544254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:17.643062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b80af-313f-43db-a285-94aa533c1902 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:17.660726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370d6a2-4815-428b-8046-61b326ea6796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:17.661635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370d6a2-4815-428b-8046-61b326ea6796 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:17.661924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370d6a2-4815-428b-8046-61b326ea6796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:17.662202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370d6a2-4815-428b-8046-61b326ea6796 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:17.662876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370d6a2-4815-428b-8046-61b326ea6796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:17.665143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370d6a2-4815-428b-8046-61b326ea6796 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:17.707984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 824/1665] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:17 2022] POST /identity/v3/users => generated 285 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:17.712991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db8b54-5fe1-4352-accd-e6dbf9081de4 service nova] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:17.728144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db8b54-5fe1-4352-accd-e6dbf9081de4 service nova] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:17.741909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05c9a3-4a48-45a9-abbc-afbc9a697306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:17.755271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05c9a3-4a48-45a9-abbc-afbc9a697306 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:17.755756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05c9a3-4a48-45a9-abbc-afbc9a697306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:17.756215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05c9a3-4a48-45a9-abbc-afbc9a697306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:17.757089 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05c9a3-4a48-45a9-abbc-afbc9a697306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:17.760283 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05c9a3-4a48-45a9-abbc-afbc9a697306 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:17.801070 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 842/1666] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:17.803814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370d6a2-4815-428b-8046-61b326ea6796 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:17.827990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370d6a2-4815-428b-8046-61b326ea6796 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:17.843632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a85bb-e1b4-4638-90f0-e13b754eaea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:17.844680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a85bb-e1b4-4638-90f0-e13b754eaea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:17.845172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a85bb-e1b4-4638-90f0-e13b754eaea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:17.845669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a85bb-e1b4-4638-90f0-e13b754eaea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:17.846558 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a85bb-e1b4-4638-90f0-e13b754eaea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:17.848477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a85bb-e1b4-4638-90f0-e13b754eaea7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:17.907369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 825/1667] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:17.908394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05c9a3-4a48-45a9-abbc-afbc9a697306 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:17.959120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05c9a3-4a48-45a9-abbc-afbc9a697306 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:17.981458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07413a8-8be1-45e8-8b9d-8be96a7d9449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:17.981708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07413a8-8be1-45e8-8b9d-8be96a7d9449 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:17.982033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07413a8-8be1-45e8-8b9d-8be96a7d9449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:17.983451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07413a8-8be1-45e8-8b9d-8be96a7d9449 admin admin] PATH_INFO: `/v3/projects/5f7a018f08f5429bb84c9b4ac3812a00/users/c28086759b9a4aee8abb45fb2237b2e9/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:18.046870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a85bb-e1b4-4638-90f0-e13b754eaea7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:18.052486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07413a8-8be1-45e8-8b9d-8be96a7d9449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7a018f08f5429bb84c9b4ac3812a00, user_id=c28086759b9a4aee8abb45fb2237b2e9, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:18.061245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07413a8-8be1-45e8-8b9d-8be96a7d9449 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:18.064958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a85bb-e1b4-4638-90f0-e13b754eaea7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:18.073091 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c4587-cfc4-4694-8315-c0db4dbd94f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:18.074463 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c4587-cfc4-4694-8315-c0db4dbd94f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:18.074930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c4587-cfc4-4694-8315-c0db4dbd94f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:18.075586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c4587-cfc4-4694-8315-c0db4dbd94f5 admin admin] PATH_INFO: `/v3/users/e5faa2cd2f4c4c52902d1c21f3388377` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:18.078250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c4587-cfc4-4694-8315-c0db4dbd94f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5faa2cd2f4c4c52902d1c21f3388377)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:18.081745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c4587-cfc4-4694-8315-c0db4dbd94f5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:18.113609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 843/1668] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:17 2022] PUT /identity/v3/projects/5f7a018f08f5429bb84c9b4ac3812a00/users/c28086759b9a4aee8abb45fb2237b2e9/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:18.126804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37302ad6-731f-4613-af93-c0a64b4ea045 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:18.127163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37302ad6-731f-4613-af93-c0a64b4ea045 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:18.127532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37302ad6-731f-4613-af93-c0a64b4ea045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:18.191425 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3b6c4587-cfc4-4694-8315-c0db4dbd94f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5faa2cd2f4c4c52902d1c21f3388377'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:18.207918 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-37302ad6-731f-4613-af93-c0a64b4ea045 None None] MFA Rules not processed for user `c28086759b9a4aee8abb45fb2237b2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:18.211547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-37302ad6-731f-4613-af93-c0a64b4ea045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:18.227474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3b6c4587-cfc4-4694-8315-c0db4dbd94f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5faa2cd2f4c4c52902d1c21f3388377'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:18.247517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3b6c4587-cfc4-4694-8315-c0db4dbd94f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5faa2cd2f4c4c52902d1c21f3388377'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:18.249968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3b6c4587-cfc4-4694-8315-c0db4dbd94f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5faa2cd2f4c4c52902d1c21f3388377'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:18.267637 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3b6c4587-cfc4-4694-8315-c0db4dbd94f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5faa2cd2f4c4c52902d1c21f3388377'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:18.284348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 826/1669] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:18 2022] DELETE /identity/v3/users/e5faa2cd2f4c4c52902d1c21f3388377 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:18.287665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331249-21e2-4ff5-a5a9-a2a10d97d7bd None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:18.287878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331249-21e2-4ff5-a5a9-a2a10d97d7bd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:18.290261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331249-21e2-4ff5-a5a9-a2a10d97d7bd None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:18.292685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 827/1670] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 22 17:07:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:07:18.296837 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaaeaf-e988-4f46-aa66-27ea8b7bf67c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:18.298399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaaeaf-e988-4f46-aa66-27ea8b7bf67c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:18.298768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaaeaf-e988-4f46-aa66-27ea8b7bf67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:18.356387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4dbaaeaf-e988-4f46-aa66-27ea8b7bf67c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:18.358514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4dbaaeaf-e988-4f46-aa66-27ea8b7bf67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:18.391642 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 844/1671] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:18.393391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9ff79-bbaf-4112-8fee-6c7fc1c58c8d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:18.393942 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9ff79-bbaf-4112-8fee-6c7fc1c58c8d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:18.394376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9ff79-bbaf-4112-8fee-6c7fc1c58c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:18.453615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-caf9ff79-bbaf-4112-8fee-6c7fc1c58c8d None None] MFA Rules not processed for user `8c2ed1eb664348e384824f39ca90c950`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:18.455759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-caf9ff79-bbaf-4112-8fee-6c7fc1c58c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:18.491418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:18.549473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 828/1672] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:18.553177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaaeaf-e988-4f46-aa66-27ea8b7bf67c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:18.556014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4dbaaeaf-e988-4f46-aa66-27ea8b7bf67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:18.587067 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaaeaf-e988-4f46-aa66-27ea8b7bf67c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:18.596860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e1665-b34c-4463-a5db-6c64aaef1630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:18.598405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e1665-b34c-4463-a5db-6c64aaef1630 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:18.599396 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e1665-b34c-4463-a5db-6c64aaef1630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:18.599779 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e1665-b34c-4463-a5db-6c64aaef1630 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:18.603685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e1665-b34c-4463-a5db-6c64aaef1630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:18.606023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e1665-b34c-4463-a5db-6c64aaef1630 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:18.619474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 845/1673] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:07:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:18.636202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9ff79-bbaf-4112-8fee-6c7fc1c58c8d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:18.641098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-caf9ff79-bbaf-4112-8fee-6c7fc1c58c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:18.676476 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 829/1674] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:18 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:18.683799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e1665-b34c-4463-a5db-6c64aaef1630 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:18.692450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:18.695468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9ff79-bbaf-4112-8fee-6c7fc1c58c8d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:18.912585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e1665-b34c-4463-a5db-6c64aaef1630 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:18.924798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27302676-8563-406b-971d-bc4ed973c249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:18.927025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302676-8563-406b-971d-bc4ed973c249 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:18.927329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302676-8563-406b-971d-bc4ed973c249 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:18.930432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302676-8563-406b-971d-bc4ed973c249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:18.942306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dab3c-dfc0-499f-bd42-4fb4af9003df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:18.943990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dab3c-dfc0-499f-bd42-4fb4af9003df admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:18.946937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dab3c-dfc0-499f-bd42-4fb4af9003df admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:18.952171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dab3c-dfc0-499f-bd42-4fb4af9003df admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:18.953827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dab3c-dfc0-499f-bd42-4fb4af9003df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:18.961478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dab3c-dfc0-499f-bd42-4fb4af9003df admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:18.965014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27302676-8563-406b-971d-bc4ed973c249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:18.972172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27302676-8563-406b-971d-bc4ed973c249 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:18.996634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 846/1675] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:07:18.997599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27302676-8563-406b-971d-bc4ed973c249 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:19.022494 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27302676-8563-406b-971d-bc4ed973c249 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:19.038026 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 830/1676] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:18 2022] POST /identity/v3/users => generated 291 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:19.040466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dab3c-dfc0-499f-bd42-4fb4af9003df admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:19.052411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7c53a-6ca6-48f3-8f7f-696d60b32f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:19.053967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7c53a-6ca6-48f3-8f7f-696d60b32f7f service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:19.055420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7c53a-6ca6-48f3-8f7f-696d60b32f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:19.057173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7c53a-6ca6-48f3-8f7f-696d60b32f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:19.093221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dab3c-dfc0-499f-bd42-4fb4af9003df admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:19.114632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7c53a-6ca6-48f3-8f7f-696d60b32f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:19.118020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7c53a-6ca6-48f3-8f7f-696d60b32f7f service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:19.132201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e98aa-7007-4549-912e-b7f2c6db3884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:19.133199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e98aa-7007-4549-912e-b7f2c6db3884 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:19.133568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e98aa-7007-4549-912e-b7f2c6db3884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:19.133926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e98aa-7007-4549-912e-b7f2c6db3884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:19.135059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e98aa-7007-4549-912e-b7f2c6db3884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:19.139992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e98aa-7007-4549-912e-b7f2c6db3884 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:19.154527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 847/1677] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:19.256440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 831/1678] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:07:19.257303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7c53a-6ca6-48f3-8f7f-696d60b32f7f service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:19.280136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7c53a-6ca6-48f3-8f7f-696d60b32f7f service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:19.292376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fddf74-a130-4447-a274-8cc004290c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:19.293160 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fddf74-a130-4447-a274-8cc004290c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:19.293530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fddf74-a130-4447-a274-8cc004290c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:19.293939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fddf74-a130-4447-a274-8cc004290c12 admin admin] PATH_INFO: `/v3/projects/28aa89927d004c14a6edddc6e7808e5b/users/c57f6e74f6a84cfea6afca1f831a2013/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:19.344498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fddf74-a130-4447-a274-8cc004290c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28aa89927d004c14a6edddc6e7808e5b, user_id=c57f6e74f6a84cfea6afca1f831a2013, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:19.347291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fddf74-a130-4447-a274-8cc004290c12 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:19.378725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 848/1679] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:19 2022] PUT /identity/v3/projects/28aa89927d004c14a6edddc6e7808e5b/users/c57f6e74f6a84cfea6afca1f831a2013/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:07:19.387133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e98aa-7007-4549-912e-b7f2c6db3884 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:19.443683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e98aa-7007-4549-912e-b7f2c6db3884 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:19.603621 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82052e-c7df-48cd-b631-1d4fe3cf1fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:19.605565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82052e-c7df-48cd-b631-1d4fe3cf1fdb admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:19.606588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82052e-c7df-48cd-b631-1d4fe3cf1fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:19.607292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82052e-c7df-48cd-b631-1d4fe3cf1fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:19.608243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82052e-c7df-48cd-b631-1d4fe3cf1fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:19.614041 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82052e-c7df-48cd-b631-1d4fe3cf1fdb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:19.633550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 832/1680] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:19.639882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fddf74-a130-4447-a274-8cc004290c12 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:19.675915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fddf74-a130-4447-a274-8cc004290c12 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:19.693139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982ee3e-c1d4-4dd1-9a11-416651df547a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:19.694307 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982ee3e-c1d4-4dd1-9a11-416651df547a admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:19.694783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982ee3e-c1d4-4dd1-9a11-416651df547a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:19.695342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982ee3e-c1d4-4dd1-9a11-416651df547a admin admin] PATH_INFO: `/v3/projects/28aa89927d004c14a6edddc6e7808e5b/users/c57f6e74f6a84cfea6afca1f831a2013/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:19.704670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982ee3e-c1d4-4dd1-9a11-416651df547a admin admin] RBAC: Authorizing `identity:create_grant(project_id=28aa89927d004c14a6edddc6e7808e5b, user_id=c57f6e74f6a84cfea6afca1f831a2013, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:19.709350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982ee3e-c1d4-4dd1-9a11-416651df547a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:19.741809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 849/1681] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:19 2022] PUT /identity/v3/projects/28aa89927d004c14a6edddc6e7808e5b/users/c57f6e74f6a84cfea6afca1f831a2013/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:19.761489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25645f-f909-4d8a-9154-65142c9a5ced None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:19.761842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25645f-f909-4d8a-9154-65142c9a5ced None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:19.762120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25645f-f909-4d8a-9154-65142c9a5ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:19.832815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3e25645f-f909-4d8a-9154-65142c9a5ced None None] MFA Rules not processed for user `c57f6e74f6a84cfea6afca1f831a2013`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:19.834132 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3e25645f-f909-4d8a-9154-65142c9a5ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:20.020490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 833/1682] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:20.028936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982ee3e-c1d4-4dd1-9a11-416651df547a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:20.031586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8982ee3e-c1d4-4dd1-9a11-416651df547a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:20.264514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982ee3e-c1d4-4dd1-9a11-416651df547a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:20.293108 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5fb18-56f3-474d-8eec-85c895b9401e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:20.294407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5fb18-56f3-474d-8eec-85c895b9401e service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:20.295683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5fb18-56f3-474d-8eec-85c895b9401e service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:20.296198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5fb18-56f3-474d-8eec-85c895b9401e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:20.358919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5fb18-56f3-474d-8eec-85c895b9401e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:20.362119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5fb18-56f3-474d-8eec-85c895b9401e service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:20.393517 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 850/1683] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:20.537388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25645f-f909-4d8a-9154-65142c9a5ced None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:20.551472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:20.589238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25645f-f909-4d8a-9154-65142c9a5ced None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:20.603830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c946a-afea-4acf-8adb-eab599043678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:20.606558 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c946a-afea-4acf-8adb-eab599043678 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:20.608710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c946a-afea-4acf-8adb-eab599043678 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:20.609217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c946a-afea-4acf-8adb-eab599043678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:20.823388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c946a-afea-4acf-8adb-eab599043678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:20.825458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c946a-afea-4acf-8adb-eab599043678 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:20.858127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 834/1684] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:21.113559 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30030361-df54-4cf7-9a78-caa1d188de1f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:21.116845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30030361-df54-4cf7-9a78-caa1d188de1f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:21.117557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30030361-df54-4cf7-9a78-caa1d188de1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:21.187526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-30030361-df54-4cf7-9a78-caa1d188de1f None None] MFA Rules not processed for user `624f8fd9a74e48afa2f8b74b090a8dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:21.189113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-30030361-df54-4cf7-9a78-caa1d188de1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:21.239428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:21.378558 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 851/1685] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:21.395293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c946a-afea-4acf-8adb-eab599043678 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:21.397170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-637c946a-afea-4acf-8adb-eab599043678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:21.628874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30030361-df54-4cf7-9a78-caa1d188de1f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:21.630360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-30030361-df54-4cf7-9a78-caa1d188de1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:21.722076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c946a-afea-4acf-8adb-eab599043678 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:21.765002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26c0bf-ed35-460e-b28c-efe521717f35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:21.765594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26c0bf-ed35-460e-b28c-efe521717f35 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:21.766472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26c0bf-ed35-460e-b28c-efe521717f35 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:21.766706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26c0bf-ed35-460e-b28c-efe521717f35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:21.782388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26c0bf-ed35-460e-b28c-efe521717f35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:21.789407 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26c0bf-ed35-460e-b28c-efe521717f35 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:21.829806 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30030361-df54-4cf7-9a78-caa1d188de1f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:21.834154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 835/1686] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:07:21.841954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:21.842657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:21.842937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:21.843295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] PATH_INFO: `/v3/projects/9c704cc59df6456ea25ca05356b23384` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:21.846095 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c704cc59df6456ea25ca05356b23384)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:21.848061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:21.969855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c704cc59df6456ea25ca05356b23384', 'request_id': 'req-26d62a03-6a79-4ac3-83ec-5488c10ad58f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:21.984105 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c704cc59df6456ea25ca05356b23384', 'request_id': 'req-26d62a03-6a79-4ac3-83ec-5488c10ad58f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:22.003505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c704cc59df6456ea25ca05356b23384', 'request_id': 'req-26d62a03-6a79-4ac3-83ec-5488c10ad58f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:22.044441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] The token cache is being invalidate because project 9c704cc59df6456ea25ca05356b23384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:22.046845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:22.048130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:22.054811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 852/1687] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:21 2022] DELETE /identity/v3/projects/9c704cc59df6456ea25ca05356b23384 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:22.055843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362275c-1ee0-4242-a905-0b5fd22bcb60 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:22.056131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362275c-1ee0-4242-a905-0b5fd22bcb60 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:22.056405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362275c-1ee0-4242-a905-0b5fd22bcb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:22.124691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7362275c-1ee0-4242-a905-0b5fd22bcb60 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:22.126004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7362275c-1ee0-4242-a905-0b5fd22bcb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:22.195789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:22.197449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:22.430299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 836/1688] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:22.443223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42bcc2-cb56-4a54-bcbd-60754cb17bed None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:22.444539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42bcc2-cb56-4a54-bcbd-60754cb17bed None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:22.444981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42bcc2-cb56-4a54-bcbd-60754cb17bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:22.448227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:22.489099 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ec42bcc2-cb56-4a54-bcbd-60754cb17bed None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:22.499686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ec42bcc2-cb56-4a54-bcbd-60754cb17bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:22.505099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d62a03-6a79-4ac3-83ec-5488c10ad58f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:22.523490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5852b-d117-45ee-8fbf-d1fdfcb03853 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:22.524728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5852b-d117-45ee-8fbf-d1fdfcb03853 service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:22.528421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5852b-d117-45ee-8fbf-d1fdfcb03853 service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:22.529005 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5852b-d117-45ee-8fbf-d1fdfcb03853 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:22.530091 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 837/1689] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:22.535154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42bcc2-cb56-4a54-bcbd-60754cb17bed None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:22.539433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c1d5852b-d117-45ee-8fbf-d1fdfcb03853 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:22.541885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ec42bcc2-cb56-4a54-bcbd-60754cb17bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:22.577953 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5852b-d117-45ee-8fbf-d1fdfcb03853 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:22.582275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42bcc2-cb56-4a54-bcbd-60754cb17bed None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:22.584924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5852b-d117-45ee-8fbf-d1fdfcb03853 service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:22.593778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a6fcf-5da9-4727-91a8-e71d110e97a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:22.597347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6fcf-5da9-4727-91a8-e71d110e97a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:22.598723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6fcf-5da9-4727-91a8-e71d110e97a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:22.599106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6fcf-5da9-4727-91a8-e71d110e97a8 admin admin] PATH_INFO: `/v3/users/a716e14c35014a728f0dc76675ff6874` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:22.604315 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a6fcf-5da9-4727-91a8-e71d110e97a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a716e14c35014a728f0dc76675ff6874)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:22.609259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a6fcf-5da9-4727-91a8-e71d110e97a8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:22.623869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 853/1690] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:07:22.624845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5852b-d117-45ee-8fbf-d1fdfcb03853 service placement] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:22.662984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5852b-d117-45ee-8fbf-d1fdfcb03853 service placement] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:22.666687 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:22.673375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cc6f1-a284-4b72-ac15-313004bde748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:22.674110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cc6f1-a284-4b72-ac15-313004bde748 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:22.674392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cc6f1-a284-4b72-ac15-313004bde748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:22.674664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cc6f1-a284-4b72-ac15-313004bde748 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:22.675628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cc6f1-a284-4b72-ac15-313004bde748 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:22.675940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cc6f1-a284-4b72-ac15-313004bde748 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:22.677642 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cc6f1-a284-4b72-ac15-313004bde748 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:22.744192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 854/1691] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:22.759776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cc6f1-a284-4b72-ac15-313004bde748 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:22.801225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cc6f1-a284-4b72-ac15-313004bde748 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:22.823385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c21086-3514-4ee2-a11d-674e8095fbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:22.823939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c21086-3514-4ee2-a11d-674e8095fbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:22.824223 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c21086-3514-4ee2-a11d-674e8095fbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:22.824603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c21086-3514-4ee2-a11d-674e8095fbc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:22.825577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c21086-3514-4ee2-a11d-674e8095fbc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:22.828295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-801a6fcf-5da9-4727-91a8-e71d110e97a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a716e14c35014a728f0dc76675ff6874'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:22.828603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c21086-3514-4ee2-a11d-674e8095fbc2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:22.909928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 855/1692] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:22 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:22.910077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-801a6fcf-5da9-4727-91a8-e71d110e97a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a716e14c35014a728f0dc76675ff6874'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:22.911535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c21086-3514-4ee2-a11d-674e8095fbc2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:22.925690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-801a6fcf-5da9-4727-91a8-e71d110e97a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a716e14c35014a728f0dc76675ff6874'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:22.927223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-801a6fcf-5da9-4727-91a8-e71d110e97a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a716e14c35014a728f0dc76675ff6874'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:22.946642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-801a6fcf-5da9-4727-91a8-e71d110e97a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a716e14c35014a728f0dc76675ff6874'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:22.975551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 838/1693] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:22 2022] DELETE /identity/v3/users/a716e14c35014a728f0dc76675ff6874 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:22.975551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb792fe0-22c5-4c18-9cba-8eb6e683c48e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:22.975551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb792fe0-22c5-4c18-9cba-8eb6e683c48e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:22.975551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb792fe0-22c5-4c18-9cba-8eb6e683c48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:23.024014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-cb792fe0-22c5-4c18-9cba-8eb6e683c48e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:23.027567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cb792fe0-22c5-4c18-9cba-8eb6e683c48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:23.151051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c21086-3514-4ee2-a11d-674e8095fbc2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:23.151731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 839/1694] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:23.177375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005aac8-b11b-4314-8378-8410b459eed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:23.178656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005aac8-b11b-4314-8378-8410b459eed2 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:23.178656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005aac8-b11b-4314-8378-8410b459eed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:23.178656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005aac8-b11b-4314-8378-8410b459eed2 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:23.179435 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005aac8-b11b-4314-8378-8410b459eed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:23.181441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005aac8-b11b-4314-8378-8410b459eed2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:23.244799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 856/1695] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:22 2022] POST /identity/v3/users => generated 284 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:07:23.249700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb792fe0-22c5-4c18-9cba-8eb6e683c48e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:23.251370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cb792fe0-22c5-4c18-9cba-8eb6e683c48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:23.283638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb792fe0-22c5-4c18-9cba-8eb6e683c48e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:23.306815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df260960-3cb3-4061-8677-51d5331b9a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:23.308038 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df260960-3cb3-4061-8677-51d5331b9a3f admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:23.309070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df260960-3cb3-4061-8677-51d5331b9a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:23.309365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df260960-3cb3-4061-8677-51d5331b9a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:23.310663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df260960-3cb3-4061-8677-51d5331b9a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:23.317489 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005aac8-b11b-4314-8378-8410b459eed2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:23.320325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c005aac8-b11b-4314-8378-8410b459eed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:23.321444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df260960-3cb3-4061-8677-51d5331b9a3f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:23.368258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 840/1696] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:23.370237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df260960-3cb3-4061-8677-51d5331b9a3f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:23.397294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df260960-3cb3-4061-8677-51d5331b9a3f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:23.423196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d9b95-665a-437d-9eb5-4bac75a53f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:23.423986 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d9b95-665a-437d-9eb5-4bac75a53f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:23.425867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d9b95-665a-437d-9eb5-4bac75a53f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:23.427629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d9b95-665a-437d-9eb5-4bac75a53f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:23.431265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d9b95-665a-437d-9eb5-4bac75a53f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:23.437791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d9b95-665a-437d-9eb5-4bac75a53f89 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:23.488908 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 841/1697] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:23.496353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d9b95-665a-437d-9eb5-4bac75a53f89 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:23.514934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005aac8-b11b-4314-8378-8410b459eed2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:23.525477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d9b95-665a-437d-9eb5-4bac75a53f89 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:23.543503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fb561-1d8b-4fa4-9e21-e7b4cc9a886d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:23.545476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fb561-1d8b-4fa4-9e21-e7b4cc9a886d service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:23.546124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fb561-1d8b-4fa4-9e21-e7b4cc9a886d service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:23.547310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fb561-1d8b-4fa4-9e21-e7b4cc9a886d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:23.567712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff00c1-9925-4113-b5cc-c3175e1c9330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:23.574984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff00c1-9925-4113-b5cc-c3175e1c9330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:23.575211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574fb561-1d8b-4fa4-9e21-e7b4cc9a886d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:23.575854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff00c1-9925-4113-b5cc-c3175e1c9330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:23.577032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574fb561-1d8b-4fa4-9e21-e7b4cc9a886d service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:23.577502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff00c1-9925-4113-b5cc-c3175e1c9330 admin admin] PATH_INFO: `/v3/projects/779c5dc4bcc34cae955f318ef26747a0/users/c24962d0db664f699cdf8e9c21000923/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:23.614660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 857/1698] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:23.647469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff00c1-9925-4113-b5cc-c3175e1c9330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=779c5dc4bcc34cae955f318ef26747a0, user_id=c24962d0db664f699cdf8e9c21000923, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:23.652755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff00c1-9925-4113-b5cc-c3175e1c9330 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:23.682984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 842/1699] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:23 2022] PUT /identity/v3/projects/779c5dc4bcc34cae955f318ef26747a0/users/c24962d0db664f699cdf8e9c21000923/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:23.699077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7eee0-e535-4ac8-9213-18648be32349 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:23.699563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7eee0-e535-4ac8-9213-18648be32349 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:23.700765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7eee0-e535-4ac8-9213-18648be32349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:23.838188 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5fe7eee0-e535-4ac8-9213-18648be32349 None None] MFA Rules not processed for user `c24962d0db664f699cdf8e9c21000923`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:23.843394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5fe7eee0-e535-4ac8-9213-18648be32349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:24.048631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:24.359620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 858/1700] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 662 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:24.365228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff00c1-9925-4113-b5cc-c3175e1c9330 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:24.516260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:24.573989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff00c1-9925-4113-b5cc-c3175e1c9330 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:24.625482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d0793-c5e5-484b-9578-d3f6fb6a8023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:24.630355 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d0793-c5e5-484b-9578-d3f6fb6a8023 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:24.634797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d0793-c5e5-484b-9578-d3f6fb6a8023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:24.636563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d0793-c5e5-484b-9578-d3f6fb6a8023 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:24.646612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d0793-c5e5-484b-9578-d3f6fb6a8023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:24.671965 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d0793-c5e5-484b-9578-d3f6fb6a8023 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:24.807638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7eee0-e535-4ac8-9213-18648be32349 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:24.808827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 843/1701] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:24 2022] POST /identity/v3/projects => generated 369 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:24.826027 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7eee0-e535-4ac8-9213-18648be32349 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:24.973787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af058ed1-817d-439e-a89e-11e6289a8104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:24.974432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af058ed1-817d-439e-a89e-11e6289a8104 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:24.974708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af058ed1-817d-439e-a89e-11e6289a8104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:24.975001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af058ed1-817d-439e-a89e-11e6289a8104 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:24.975659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af058ed1-817d-439e-a89e-11e6289a8104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:24.982621 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af058ed1-817d-439e-a89e-11e6289a8104 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:25.058016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 859/1702] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:24 2022] POST /identity/v3/users => generated 291 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:25.059333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d0793-c5e5-484b-9578-d3f6fb6a8023 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:25.127190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d0793-c5e5-484b-9578-d3f6fb6a8023 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:25.162194 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99560ab-c9d9-4d6b-bd24-113cac0160c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:25.170688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99560ab-c9d9-4d6b-bd24-113cac0160c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:25.172263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99560ab-c9d9-4d6b-bd24-113cac0160c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:25.173100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99560ab-c9d9-4d6b-bd24-113cac0160c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:25.175711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99560ab-c9d9-4d6b-bd24-113cac0160c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:25.181971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99560ab-c9d9-4d6b-bd24-113cac0160c0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:25.290315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af058ed1-817d-439e-a89e-11e6289a8104 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:25.303072 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 844/1703] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:25.316121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af058ed1-817d-439e-a89e-11e6289a8104 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:25.334256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f7fac-881b-4355-8856-a8077b3c883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:25.335932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7fac-881b-4355-8856-a8077b3c883b admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:25.337638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7fac-881b-4355-8856-a8077b3c883b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:25.339815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7fac-881b-4355-8856-a8077b3c883b admin admin] PATH_INFO: `/v3/projects/79533dd9ee05460e94bd5b9ed433f361/users/dbb753168ee640ba808195e135022fff/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:25.397390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f7fac-881b-4355-8856-a8077b3c883b admin admin] RBAC: Authorizing `identity:create_grant(project_id=79533dd9ee05460e94bd5b9ed433f361, user_id=dbb753168ee640ba808195e135022fff, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:25.400381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f7fac-881b-4355-8856-a8077b3c883b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:25.451645 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 860/1704] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:25 2022] PUT /identity/v3/projects/79533dd9ee05460e94bd5b9ed433f361/users/dbb753168ee640ba808195e135022fff/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:25.455942 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99560ab-c9d9-4d6b-bd24-113cac0160c0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:25.498580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99560ab-c9d9-4d6b-bd24-113cac0160c0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:25.677865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064302a1-26c2-4065-945e-a722ebd83549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:25.678886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064302a1-26c2-4065-945e-a722ebd83549 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:25.678886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064302a1-26c2-4065-945e-a722ebd83549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:25.679536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064302a1-26c2-4065-945e-a722ebd83549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:25.680314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064302a1-26c2-4065-945e-a722ebd83549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:25.689130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064302a1-26c2-4065-945e-a722ebd83549 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:25.728754 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f7fac-881b-4355-8856-a8077b3c883b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:25.731172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 845/1705] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:25.746655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f7fac-881b-4355-8856-a8077b3c883b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:25.766194 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe057b-e4ef-4c76-9346-81ddf148ed86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:25.767037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe057b-e4ef-4c76-9346-81ddf148ed86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:25.767458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe057b-e4ef-4c76-9346-81ddf148ed86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:25.767931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe057b-e4ef-4c76-9346-81ddf148ed86 admin admin] PATH_INFO: `/v3/projects/79533dd9ee05460e94bd5b9ed433f361/users/dbb753168ee640ba808195e135022fff/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:25.775832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe057b-e4ef-4c76-9346-81ddf148ed86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79533dd9ee05460e94bd5b9ed433f361, user_id=dbb753168ee640ba808195e135022fff, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:25.780412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe057b-e4ef-4c76-9346-81ddf148ed86 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:25.819528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c0ec-245f-4967-9fdb-d9c93ffbd80d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:25.820777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c0ec-245f-4967-9fdb-d9c93ffbd80d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:25.820777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c0ec-245f-4967-9fdb-d9c93ffbd80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:25.823565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 861/1706] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:25 2022] PUT /identity/v3/projects/79533dd9ee05460e94bd5b9ed433f361/users/dbb753168ee640ba808195e135022fff/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:07:25.893367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5e12c0ec-245f-4967-9fdb-d9c93ffbd80d None None] MFA Rules not processed for user `dbb753168ee640ba808195e135022fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:25.895932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5e12c0ec-245f-4967-9fdb-d9c93ffbd80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:26.146373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 846/1707] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:26.177487 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe057b-e4ef-4c76-9346-81ddf148ed86 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:26.181238 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-97fe057b-e4ef-4c76-9346-81ddf148ed86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:26.456589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe057b-e4ef-4c76-9346-81ddf148ed86 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:26.473274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa974902-4b53-4cae-bc47-017c0b795f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:26.474284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa974902-4b53-4cae-bc47-017c0b795f7f service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:26.474824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa974902-4b53-4cae-bc47-017c0b795f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:26.475400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa974902-4b53-4cae-bc47-017c0b795f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:26.500597 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa974902-4b53-4cae-bc47-017c0b795f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:26.503653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa974902-4b53-4cae-bc47-017c0b795f7f service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:26.545236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 862/1708] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:26.990166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12c0ec-245f-4967-9fdb-d9c93ffbd80d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:26.995778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5e12c0ec-245f-4967-9fdb-d9c93ffbd80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:27.188325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:27.188696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa974902-4b53-4cae-bc47-017c0b795f7f service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:27.190745 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-aa974902-4b53-4cae-bc47-017c0b795f7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:27.281139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12c0ec-245f-4967-9fdb-d9c93ffbd80d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:27.292102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:27.300194 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:27.302529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:27.302805 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:27.305451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] PATH_INFO: `/v3/projects/329d492f457d4bc6bf63ff3eb5971826` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:27.309612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=329d492f457d4bc6bf63ff3eb5971826)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:27.316955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:27.468822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329d492f457d4bc6bf63ff3eb5971826', 'request_id': 'req-3e3251d6-91a3-4a39-9c22-b01f71d24f44'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:27.484788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329d492f457d4bc6bf63ff3eb5971826', 'request_id': 'req-3e3251d6-91a3-4a39-9c22-b01f71d24f44'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:27.500565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329d492f457d4bc6bf63ff3eb5971826', 'request_id': 'req-3e3251d6-91a3-4a39-9c22-b01f71d24f44'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:27.539121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] The token cache is being invalidate because project 329d492f457d4bc6bf63ff3eb5971826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:27.539590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:27.545611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa974902-4b53-4cae-bc47-017c0b795f7f service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:27.547376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3e3251d6-91a3-4a39-9c22-b01f71d24f44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:27.555582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 847/1709] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:26 2022] DELETE /identity/v3/projects/329d492f457d4bc6bf63ff3eb5971826 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:27.562536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9edaeb-fba0-4ce8-b37f-6e18df259e55 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:27.564036 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9edaeb-fba0-4ce8-b37f-6e18df259e55 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:27.565265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9edaeb-fba0-4ce8-b37f-6e18df259e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:27.657307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bb9edaeb-fba0-4ce8-b37f-6e18df259e55 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:27.657307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bb9edaeb-fba0-4ce8-b37f-6e18df259e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:27.803903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f23e7f-bf0e-4c31-a55f-9a171d78c25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:27.818026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f23e7f-bf0e-4c31-a55f-9a171d78c25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:27.819248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f23e7f-bf0e-4c31-a55f-9a171d78c25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:27.820157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f23e7f-bf0e-4c31-a55f-9a171d78c25b admin admin] PATH_INFO: `/v3/users/c28086759b9a4aee8abb45fb2237b2e9` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:27.825411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f23e7f-bf0e-4c31-a55f-9a171d78c25b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c28086759b9a4aee8abb45fb2237b2e9)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:27.840960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f23e7f-bf0e-4c31-a55f-9a171d78c25b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:27.992302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f3f23e7f-bf0e-4c31-a55f-9a171d78c25b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c28086759b9a4aee8abb45fb2237b2e9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:28.015546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 848/1710] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:28.027296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31154232-817a-4bc0-b2c6-f827669a77b7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:28.028180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31154232-817a-4bc0-b2c6-f827669a77b7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:28.031569 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31154232-817a-4bc0-b2c6-f827669a77b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:28.039483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f3f23e7f-bf0e-4c31-a55f-9a171d78c25b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c28086759b9a4aee8abb45fb2237b2e9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:28.069161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f3f23e7f-bf0e-4c31-a55f-9a171d78c25b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c28086759b9a4aee8abb45fb2237b2e9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:28.089031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f3f23e7f-bf0e-4c31-a55f-9a171d78c25b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c28086759b9a4aee8abb45fb2237b2e9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:28.089031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f3f23e7f-bf0e-4c31-a55f-9a171d78c25b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c28086759b9a4aee8abb45fb2237b2e9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:28.118674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 863/1711] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:27 2022] DELETE /identity/v3/users/c28086759b9a4aee8abb45fb2237b2e9 => generated 0 bytes in 931 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:28.136508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689a8dd-8b9f-4c1b-98df-bcf9eddf5b0d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:28.137113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689a8dd-8b9f-4c1b-98df-bcf9eddf5b0d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:28.137730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689a8dd-8b9f-4c1b-98df-bcf9eddf5b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:28.195201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-31154232-817a-4bc0-b2c6-f827669a77b7 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:28.195201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-31154232-817a-4bc0-b2c6-f827669a77b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:28.214218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e689a8dd-8b9f-4c1b-98df-bcf9eddf5b0d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:28.219001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e689a8dd-8b9f-4c1b-98df-bcf9eddf5b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:28.374609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:28.427576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 864/1712] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:28.429292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9876a-a3f0-4195-be84-473ac8458c8b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:28.429587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9876a-a3f0-4195-be84-473ac8458c8b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:28.430325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9876a-a3f0-4195-be84-473ac8458c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:28.540395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 849/1713] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:28.549727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31154232-817a-4bc0-b2c6-f827669a77b7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:28.558436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-31154232-817a-4bc0-b2c6-f827669a77b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:28.601850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2ba9876a-a3f0-4195-be84-473ac8458c8b None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:28.604239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2ba9876a-a3f0-4195-be84-473ac8458c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:28.621365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:28.763211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31154232-817a-4bc0-b2c6-f827669a77b7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:28.779066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266b047-9da3-45f1-9d4d-64368a20231b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:28.780096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266b047-9da3-45f1-9d4d-64368a20231b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:28.782064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266b047-9da3-45f1-9d4d-64368a20231b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:28.782482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266b047-9da3-45f1-9d4d-64368a20231b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:28.818092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266b047-9da3-45f1-9d4d-64368a20231b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:28.822434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266b047-9da3-45f1-9d4d-64368a20231b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:28.858192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 850/1714] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:28.860280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 865/1715] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:07:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:28.880079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90812077-5829-4bf0-bd32-d62d00e16470 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:28.882428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9876a-a3f0-4195-be84-473ac8458c8b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:28.882998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90812077-5829-4bf0-bd32-d62d00e16470 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:28.884720 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90812077-5829-4bf0-bd32-d62d00e16470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:28.948223 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9876a-a3f0-4195-be84-473ac8458c8b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:28.958221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80d471-372a-4257-9dd9-d52981435d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:28.959135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80d471-372a-4257-9dd9-d52981435d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:28.959602 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80d471-372a-4257-9dd9-d52981435d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:28.959953 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80d471-372a-4257-9dd9-d52981435d5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:28.960708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80d471-372a-4257-9dd9-d52981435d5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:28.961665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80d471-372a-4257-9dd9-d52981435d5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:28.963666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80d471-372a-4257-9dd9-d52981435d5b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:28.985807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-90812077-5829-4bf0-bd32-d62d00e16470 None None] MFA Rules not processed for user `b34e9318826c49ed891dde1f7782898b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:28.989122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-90812077-5829-4bf0-bd32-d62d00e16470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:28.992196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 866/1716] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:29.000247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80d471-372a-4257-9dd9-d52981435d5b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:29.030089 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80d471-372a-4257-9dd9-d52981435d5b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:29.045292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf717e6-d3e4-4b3d-92c8-e2fe59427b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:29.045292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf717e6-d3e4-4b3d-92c8-e2fe59427b91 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:29.045292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf717e6-d3e4-4b3d-92c8-e2fe59427b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:29.045700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf717e6-d3e4-4b3d-92c8-e2fe59427b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:29.046415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf717e6-d3e4-4b3d-92c8-e2fe59427b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:29.052236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf717e6-d3e4-4b3d-92c8-e2fe59427b91 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:29.126353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 867/1717] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:28 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:29.132002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf717e6-d3e4-4b3d-92c8-e2fe59427b91 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:29.156277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 851/1718] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:29.172703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90812077-5829-4bf0-bd32-d62d00e16470 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:29.174185 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-90812077-5829-4bf0-bd32-d62d00e16470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:29.358394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf717e6-d3e4-4b3d-92c8-e2fe59427b91 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:29.391714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9e0a0-864f-42e1-9669-9b1ed9e593de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:29.392363 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9e0a0-864f-42e1-9669-9b1ed9e593de admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:29.392652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9e0a0-864f-42e1-9669-9b1ed9e593de admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:29.392933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9e0a0-864f-42e1-9669-9b1ed9e593de admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:29.393619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9e0a0-864f-42e1-9669-9b1ed9e593de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:29.395666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9e0a0-864f-42e1-9669-9b1ed9e593de admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:29.454231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 868/1719] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:29 2022] POST /identity/v3/users => generated 283 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:29.455411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9e0a0-864f-42e1-9669-9b1ed9e593de admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:29.462273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-77c9e0a0-864f-42e1-9669-9b1ed9e593de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:29.494865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90812077-5829-4bf0-bd32-d62d00e16470 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:29.514415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:29.675096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:29.721792 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b041d-7977-4493-94c7-0b824b9150bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:29.722590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b041d-7977-4493-94c7-0b824b9150bd service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:29.722908 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b041d-7977-4493-94c7-0b824b9150bd service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:29.723210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b041d-7977-4493-94c7-0b824b9150bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:29.728144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9e0a0-864f-42e1-9669-9b1ed9e593de admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:29.739411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a899d1-f03b-412b-9773-bea3044b758e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:29.739713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a899d1-f03b-412b-9773-bea3044b758e service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:29.740105 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a899d1-f03b-412b-9773-bea3044b758e service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:29.740485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a899d1-f03b-412b-9773-bea3044b758e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:29.880084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b041d-7977-4493-94c7-0b824b9150bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:29.881357 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b041d-7977-4493-94c7-0b824b9150bd service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:29.899677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a899d1-f03b-412b-9773-bea3044b758e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:29.902471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a899d1-f03b-412b-9773-bea3044b758e service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:29.926459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 852/1720] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 754 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:29.928133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b041d-7977-4493-94c7-0b824b9150bd service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:29.939519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 869/1721] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:07:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:07:29.952792 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b041d-7977-4493-94c7-0b824b9150bd service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:29.968565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9c366-91e4-48f3-872d-b592205d7d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:29.969383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9c366-91e4-48f3-872d-b592205d7d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:29.969885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9c366-91e4-48f3-872d-b592205d7d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:29.970083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9c366-91e4-48f3-872d-b592205d7d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:29.970760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9c366-91e4-48f3-872d-b592205d7d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:29.973097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9c366-91e4-48f3-872d-b592205d7d0c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:29.999100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 853/1722] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:30.002413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a899d1-f03b-412b-9773-bea3044b758e service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:30.032327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a899d1-f03b-412b-9773-bea3044b758e service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:30.051076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564302-3be9-4aef-a8de-f07c9a2a95a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:30.055367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564302-3be9-4aef-a8de-f07c9a2a95a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:30.057613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564302-3be9-4aef-a8de-f07c9a2a95a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:30.058245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564302-3be9-4aef-a8de-f07c9a2a95a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:30.059281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea564302-3be9-4aef-a8de-f07c9a2a95a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:30.069743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea564302-3be9-4aef-a8de-f07c9a2a95a8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:30.169146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9c366-91e4-48f3-872d-b592205d7d0c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:30.170109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 870/1723] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:30.209293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9c366-91e4-48f3-872d-b592205d7d0c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:30.266262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81884284-e022-44bf-b407-8c2b5bd3b530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:30.267391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81884284-e022-44bf-b407-8c2b5bd3b530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:30.267678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81884284-e022-44bf-b407-8c2b5bd3b530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:30.268139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81884284-e022-44bf-b407-8c2b5bd3b530 admin admin] PATH_INFO: `/v3/projects/e4fd50307b4a4e53b628a2e1ec920101/users/5e462b464e6a4231809d9fd30b44eb6d/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:30.352332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81884284-e022-44bf-b407-8c2b5bd3b530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fd50307b4a4e53b628a2e1ec920101, user_id=5e462b464e6a4231809d9fd30b44eb6d, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:30.358062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81884284-e022-44bf-b407-8c2b5bd3b530 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:30.371862 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564302-3be9-4aef-a8de-f07c9a2a95a8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:30.381261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ea564302-3be9-4aef-a8de-f07c9a2a95a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:30.434221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 854/1724] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:30 2022] PUT /identity/v3/projects/e4fd50307b4a4e53b628a2e1ec920101/users/5e462b464e6a4231809d9fd30b44eb6d/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:07:30.467082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045a8ce-b00b-4a62-bf0b-ff6496fd8f9c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:30.467082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045a8ce-b00b-4a62-bf0b-ff6496fd8f9c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:30.467082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045a8ce-b00b-4a62-bf0b-ff6496fd8f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:30.492370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564302-3be9-4aef-a8de-f07c9a2a95a8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:30.643991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b045a8ce-b00b-4a62-bf0b-ff6496fd8f9c None None] MFA Rules not processed for user `5e462b464e6a4231809d9fd30b44eb6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:30.646201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b045a8ce-b00b-4a62-bf0b-ff6496fd8f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:30.693430 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:30.694843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:30.695699 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:30.696300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] PATH_INFO: `/v3/users/c24962d0db664f699cdf8e9c21000923` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:30.702930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c24962d0db664f699cdf8e9c21000923)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:30.707203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:30.851890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:30.918330 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 855/1725] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:30.929962 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c200-8a85-49b1-99f8-59e5ede7d3df None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:30.930391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c200-8a85-49b1-99f8-59e5ede7d3df None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:30.930679 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c200-8a85-49b1-99f8-59e5ede7d3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:30.932796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c24962d0db664f699cdf8e9c21000923'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:30.974560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c24962d0db664f699cdf8e9c21000923'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:31.009093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-16f2c200-8a85-49b1-99f8-59e5ede7d3df None None] MFA Rules not processed for user `bc8c2a5413c943ed94d76eceb2720f17`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:31.010068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c24962d0db664f699cdf8e9c21000923'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:31.010399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-16f2c200-8a85-49b1-99f8-59e5ede7d3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:31.036033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c24962d0db664f699cdf8e9c21000923'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:31.039777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c24962d0db664f699cdf8e9c21000923'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:31.067392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 871/1726] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:30 2022] DELETE /identity/v3/users/c24962d0db664f699cdf8e9c21000923 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:31.082496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:31.085322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:31.099003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:31.209880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 856/1727] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:31.211418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4a836-dd9c-461a-a8a4-7d069c2990b1 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:31.211706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4a836-dd9c-461a-a8a4-7d069c2990b1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:31.211981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4a836-dd9c-461a-a8a4-7d069c2990b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:31.285347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-70f4a836-dd9c-461a-a8a4-7d069c2990b1 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:31.288330 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-70f4a836-dd9c-461a-a8a4-7d069c2990b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:31.387182 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17147879-939c-4cee-85d4-9d3c069950b9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:31.397712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17197803-c2c0-42ed-aaff-22e1e38af670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:31.398358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197803-c2c0-42ed-aaff-22e1e38af670 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:31.398634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197803-c2c0-42ed-aaff-22e1e38af670 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:31.398909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197803-c2c0-42ed-aaff-22e1e38af670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:31.412815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-17197803-c2c0-42ed-aaff-22e1e38af670 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:31.639645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 857/1728] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:31.643548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5acf51-ca3f-4491-82b4-e73ab66c1f71 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:31.644394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5acf51-ca3f-4491-82b4-e73ab66c1f71 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:31.645089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5acf51-ca3f-4491-82b4-e73ab66c1f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:31.793271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17197803-c2c0-42ed-aaff-22e1e38af670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:31.796414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17197803-c2c0-42ed-aaff-22e1e38af670 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:31.796554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8c5acf51-ca3f-4491-82b4-e73ab66c1f71 None None] MFA Rules not processed for user `bc8c2a5413c943ed94d76eceb2720f17`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:31.804725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8c5acf51-ca3f-4491-82b4-e73ab66c1f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:31.874983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 872/1729] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 790 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:31.952504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17197803-c2c0-42ed-aaff-22e1e38af670 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:31.954467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-17197803-c2c0-42ed-aaff-22e1e38af670 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:32.092446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 858/1730] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:32.110044 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5acf51-ca3f-4491-82b4-e73ab66c1f71 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:32.131355 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8c5acf51-ca3f-4491-82b4-e73ab66c1f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:32.353619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17197803-c2c0-42ed-aaff-22e1e38af670 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:32.375139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a97644-9d83-40ee-98f0-8f252fd2b7e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:32.379104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a97644-9d83-40ee-98f0-8f252fd2b7e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:32.380925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a97644-9d83-40ee-98f0-8f252fd2b7e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:32.387149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a97644-9d83-40ee-98f0-8f252fd2b7e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:32.436620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a97644-9d83-40ee-98f0-8f252fd2b7e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:32.445519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a97644-9d83-40ee-98f0-8f252fd2b7e6 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:32.498204 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5acf51-ca3f-4491-82b4-e73ab66c1f71 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:32.499766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 873/1731] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:07:32.554036 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c160b-e182-4412-a8bf-8b76172b7292 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:32.561614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c160b-e182-4412-a8bf-8b76172b7292 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:32.562148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c160b-e182-4412-a8bf-8b76172b7292 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:32.562632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c160b-e182-4412-a8bf-8b76172b7292 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:32.604141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:32.632421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c160b-e182-4412-a8bf-8b76172b7292 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:32.649212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c160b-e182-4412-a8bf-8b76172b7292 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:32.715484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 859/1732] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:33.094893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a97644-9d83-40ee-98f0-8f252fd2b7e6 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:33.097521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-79a97644-9d83-40ee-98f0-8f252fd2b7e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:33.146211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:33.192172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a97644-9d83-40ee-98f0-8f252fd2b7e6 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:33.222226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:33.224096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:33.224683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:33.234323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] PATH_INFO: `/v3/projects/5f7a018f08f5429bb84c9b4ac3812a00` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:33.239370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7a018f08f5429bb84c9b4ac3812a00)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:33.251050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:33.525946 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7a018f08f5429bb84c9b4ac3812a00', 'request_id': 'req-73d9ca28-5d59-4a3a-8592-a0e302b93d51'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:33.553046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7a018f08f5429bb84c9b4ac3812a00', 'request_id': 'req-73d9ca28-5d59-4a3a-8592-a0e302b93d51'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:33.566205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7a018f08f5429bb84c9b4ac3812a00', 'request_id': 'req-73d9ca28-5d59-4a3a-8592-a0e302b93d51'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:33.591297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] The token cache is being invalidate because project 5f7a018f08f5429bb84c9b4ac3812a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:33.591652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:33.592677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-73d9ca28-5d59-4a3a-8592-a0e302b93d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:33.597343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 874/1733] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:33 2022] DELETE /identity/v3/projects/5f7a018f08f5429bb84c9b4ac3812a00 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:33.604086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c160b-e182-4412-a8bf-8b76172b7292 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:33.663650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c160b-e182-4412-a8bf-8b76172b7292 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:33.827131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0257ceb-57fc-4c0b-88e3-12101f2bf82e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:33.828381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0257ceb-57fc-4c0b-88e3-12101f2bf82e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:33.829090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0257ceb-57fc-4c0b-88e3-12101f2bf82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:33.954840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d0257ceb-57fc-4c0b-88e3-12101f2bf82e None None] MFA Rules not processed for user `624f8fd9a74e48afa2f8b74b090a8dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:33.956675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d0257ceb-57fc-4c0b-88e3-12101f2bf82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:34.135421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:34.136484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:34.136997 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:34.137492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] PATH_INFO: `/v3/users/c57f6e74f6a84cfea6afca1f831a2013` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:34.160896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c57f6e74f6a84cfea6afca1f831a2013)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:34.167376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:34.413979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 875/1734] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 589 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:34.423488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7747c-ee99-42a8-a114-31c9368bf72d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:34.423488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7747c-ee99-42a8-a114-31c9368bf72d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:34.423771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7747c-ee99-42a8-a114-31c9368bf72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:34.496190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c57f6e74f6a84cfea6afca1f831a2013'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:34.545334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c57f6e74f6a84cfea6afca1f831a2013'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:34.551866 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5ec7747c-ee99-42a8-a114-31c9368bf72d None None] MFA Rules not processed for user `8c2ed1eb664348e384824f39ca90c950`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:34.559491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5ec7747c-ee99-42a8-a114-31c9368bf72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:34.566696 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c57f6e74f6a84cfea6afca1f831a2013'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:34.577432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c57f6e74f6a84cfea6afca1f831a2013'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:34.596906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c57f6e74f6a84cfea6afca1f831a2013'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:34.612428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 860/1735] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:33 2022] DELETE /identity/v3/users/c57f6e74f6a84cfea6afca1f831a2013 => generated 0 bytes in 1010 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:34.858923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:34.862053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:34.895659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 876/1736] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Sep 22 17:07:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:34.934172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7747c-ee99-42a8-a114-31c9368bf72d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:34.937312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5ec7747c-ee99-42a8-a114-31c9368bf72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:34.987599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7747c-ee99-42a8-a114-31c9368bf72d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:35.187627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cffd48-c966-40aa-adc2-317b5de95d52 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:35.206128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:35.206840 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:35.207171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:35.207434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] PATH_INFO: `/v3/projects/779c5dc4bcc34cae955f318ef26747a0` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:35.207743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:35.209205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] RBAC: Authorizing `identity:delete_project(project_id=779c5dc4bcc34cae955f318ef26747a0)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:35.211941 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:35.249068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a476e-16e6-4d4a-a1d3-671fdc6e445a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:35.254996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a476e-16e6-4d4a-a1d3-671fdc6e445a service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:35.256764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a476e-16e6-4d4a-a1d3-671fdc6e445a service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:35.257512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a476e-16e6-4d4a-a1d3-671fdc6e445a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:35.277297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:35.298838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a476e-16e6-4d4a-a1d3-671fdc6e445a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:35.313109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a476e-16e6-4d4a-a1d3-671fdc6e445a service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:35.409479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 877/1737] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:35.437693 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '779c5dc4bcc34cae955f318ef26747a0', 'request_id': 'req-3fc91860-a9a2-4799-aa09-77ba0c73101d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:35.456110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '779c5dc4bcc34cae955f318ef26747a0', 'request_id': 'req-3fc91860-a9a2-4799-aa09-77ba0c73101d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:35.496973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '779c5dc4bcc34cae955f318ef26747a0', 'request_id': 'req-3fc91860-a9a2-4799-aa09-77ba0c73101d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:35.585607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] The token cache is being invalidate because project 779c5dc4bcc34cae955f318ef26747a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:35.586136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:35.593518 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fc91860-a9a2-4799-aa09-77ba0c73101d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:35.607554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a476e-16e6-4d4a-a1d3-671fdc6e445a service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:35.610094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 861/1738] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:34 2022] DELETE /identity/v3/projects/779c5dc4bcc34cae955f318ef26747a0 => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:07:35.662709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a476e-16e6-4d4a-a1d3-671fdc6e445a service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:35.933567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d715e-2f9c-49ad-b463-8eff17330383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:35.935043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d715e-2f9c-49ad-b463-8eff17330383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:35.935972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d715e-2f9c-49ad-b463-8eff17330383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:35.936633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d715e-2f9c-49ad-b463-8eff17330383 admin admin] PATH_INFO: `/v3/users/dbb753168ee640ba808195e135022fff` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:35.940129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d715e-2f9c-49ad-b463-8eff17330383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbb753168ee640ba808195e135022fff)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:35.945607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d715e-2f9c-49ad-b463-8eff17330383 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:36.103746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2a2d715e-2f9c-49ad-b463-8eff17330383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbb753168ee640ba808195e135022fff'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:36.136603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2a2d715e-2f9c-49ad-b463-8eff17330383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbb753168ee640ba808195e135022fff'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:36.163612 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2a2d715e-2f9c-49ad-b463-8eff17330383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbb753168ee640ba808195e135022fff'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:36.186361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2a2d715e-2f9c-49ad-b463-8eff17330383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbb753168ee640ba808195e135022fff'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:36.190586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2a2d715e-2f9c-49ad-b463-8eff17330383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbb753168ee640ba808195e135022fff'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:36.232326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 878/1739] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:35 2022] DELETE /identity/v3/users/dbb753168ee640ba808195e135022fff => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:36.271333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b7bf-40ab-4859-8354-6f98fc5df5f5 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:36.271333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b7bf-40ab-4859-8354-6f98fc5df5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:36.271333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b7bf-40ab-4859-8354-6f98fc5df5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:36.404198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b5d8b7bf-40ab-4859-8354-6f98fc5df5f5 None None] MFA Rules not processed for user `5feea3b3b7e4401ea7db242cb60fdfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:36.407758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b5d8b7bf-40ab-4859-8354-6f98fc5df5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:36.709726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ef386-88fd-4afd-9f12-1ffae0a67aec None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:36.710321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ef386-88fd-4afd-9f12-1ffae0a67aec None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:36.710716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ef386-88fd-4afd-9f12-1ffae0a67aec None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:36.713741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 879/1740] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 22 17:07:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:07:36.739459 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2e102-cbb3-4ed6-bdc3-a9774c73f8ed None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:36.740143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2e102-cbb3-4ed6-bdc3-a9774c73f8ed None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:36.741489 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2e102-cbb3-4ed6-bdc3-a9774c73f8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:36.804400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 862/1741] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 553 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:36.854529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-18f2e102-cbb3-4ed6-bdc3-a9774c73f8ed None None] MFA Rules not processed for user `8c2ed1eb664348e384824f39ca90c950`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:36.860413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-18f2e102-cbb3-4ed6-bdc3-a9774c73f8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:37.127840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 880/1742] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:07:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:37.276780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8b7bf-40ab-4859-8354-6f98fc5df5f5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:37.279220 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b5d8b7bf-40ab-4859-8354-6f98fc5df5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:37.335461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8b7bf-40ab-4859-8354-6f98fc5df5f5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:37.350581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed370bf4-4890-4181-a932-07ce02ea3513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:37.351692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed370bf4-4890-4181-a932-07ce02ea3513 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:37.352199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed370bf4-4890-4181-a932-07ce02ea3513 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:37.352684 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed370bf4-4890-4181-a932-07ce02ea3513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:37.383627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed370bf4-4890-4181-a932-07ce02ea3513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:37.386798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed370bf4-4890-4181-a932-07ce02ea3513 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:37.427488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 863/1743] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:07:38.587648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2e102-cbb3-4ed6-bdc3-a9774c73f8ed None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:38.594940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-18f2e102-cbb3-4ed6-bdc3-a9774c73f8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:38.611504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed370bf4-4890-4181-a932-07ce02ea3513 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:38.615000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ed370bf4-4890-4181-a932-07ce02ea3513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:38.645085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:38.931487 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2e102-cbb3-4ed6-bdc3-a9774c73f8ed None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:38.960681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed370bf4-4890-4181-a932-07ce02ea3513 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:38.962774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:38.964504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:38.965220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:38.966529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] PATH_INFO: `/v3/projects/79533dd9ee05460e94bd5b9ed433f361` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:38.976296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=79533dd9ee05460e94bd5b9ed433f361)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:38.977893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:38.979885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:38.981096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:38.985457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:38.986052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] PATH_INFO: `/v3/projects/28aa89927d004c14a6edddc6e7808e5b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:38.992535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=28aa89927d004c14a6edddc6e7808e5b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:38.994811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:39.088295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:39.182405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28aa89927d004c14a6edddc6e7808e5b', 'request_id': 'req-64f6b87a-85b7-460e-aec7-1276e0c837ba'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:39.207407 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28aa89927d004c14a6edddc6e7808e5b', 'request_id': 'req-64f6b87a-85b7-460e-aec7-1276e0c837ba'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:39.228668 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28aa89927d004c14a6edddc6e7808e5b', 'request_id': 'req-64f6b87a-85b7-460e-aec7-1276e0c837ba'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:39.272507 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] The token cache is being invalidate because project 28aa89927d004c14a6edddc6e7808e5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:39.272984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:39.273854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-64f6b87a-85b7-460e-aec7-1276e0c837ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:39.275651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 864/1744] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:38 2022] DELETE /identity/v3/projects/28aa89927d004c14a6edddc6e7808e5b => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:07:39.284688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec346d3-4691-45f1-88d2-792555a38a77 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:39.285259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec346d3-4691-45f1-88d2-792555a38a77 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:39.285545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec346d3-4691-45f1-88d2-792555a38a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:39.330274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79533dd9ee05460e94bd5b9ed433f361', 'request_id': 'req-f8b650a1-3b15-464d-b231-ff71a9b69c5e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:39.334101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1ec346d3-4691-45f1-88d2-792555a38a77 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:39.334823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1ec346d3-4691-45f1-88d2-792555a38a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:39.387562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79533dd9ee05460e94bd5b9ed433f361', 'request_id': 'req-f8b650a1-3b15-464d-b231-ff71a9b69c5e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:39.434247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79533dd9ee05460e94bd5b9ed433f361', 'request_id': 'req-f8b650a1-3b15-464d-b231-ff71a9b69c5e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:39.494493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] The token cache is being invalidate because project 79533dd9ee05460e94bd5b9ed433f361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:39.495871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:39.501335 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b650a1-3b15-464d-b231-ff71a9b69c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:39.507886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 881/1745] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:38 2022] DELETE /identity/v3/projects/79533dd9ee05460e94bd5b9ed433f361 => generated 0 bytes in 921 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:39.515751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a5ee3-e0bd-4a2b-83ce-74fb1ffb9f03 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:39.516752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a5ee3-e0bd-4a2b-83ce-74fb1ffb9f03 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:39.517520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a5ee3-e0bd-4a2b-83ce-74fb1ffb9f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:39.527060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 865/1746] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:39.531706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9b5f1-6bff-4925-b77d-ea4b3d51e3d2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:39.531995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9b5f1-6bff-4925-b77d-ea4b3d51e3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:39.532272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9b5f1-6bff-4925-b77d-ea4b3d51e3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:39.593562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-31a9b5f1-6bff-4925-b77d-ea4b3d51e3d2 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:39.594852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-31a9b5f1-6bff-4925-b77d-ea4b3d51e3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:39.612271 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 866/1747] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:39.616320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9b5f1-6bff-4925-b77d-ea4b3d51e3d2 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:39.622735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-fc1a5ee3-e0bd-4a2b-83ce-74fb1ffb9f03 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:39.626322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fc1a5ee3-e0bd-4a2b-83ce-74fb1ffb9f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:39.643245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 882/1748] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:39.659169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9b5f1-6bff-4925-b77d-ea4b3d51e3d2 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:39.659681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d8d65-e735-452f-841b-cf861ae1ccc4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:39.661758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d8d65-e735-452f-841b-cf861ae1ccc4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:39.662414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d8d65-e735-452f-841b-cf861ae1ccc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:39.692037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42513f-abac-444e-abdd-044c8cba4a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:39.692037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42513f-abac-444e-abdd-044c8cba4a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:39.692037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42513f-abac-444e-abdd-044c8cba4a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:39.692037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42513f-abac-444e-abdd-044c8cba4a39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:39.692561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42513f-abac-444e-abdd-044c8cba4a39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:39.693332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42513f-abac-444e-abdd-044c8cba4a39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:39.695593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42513f-abac-444e-abdd-044c8cba4a39 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:39.744637 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 867/1749] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 17:07:39.749935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42513f-abac-444e-abdd-044c8cba4a39 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:39.778486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bb4d8d65-e735-452f-841b-cf861ae1ccc4 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:39.785164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bb4d8d65-e735-452f-841b-cf861ae1ccc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:39.793601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42513f-abac-444e-abdd-044c8cba4a39 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:39.821374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab545017-b163-42fb-b457-032468112c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:39.822402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab545017-b163-42fb-b457-032468112c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:39.822899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab545017-b163-42fb-b457-032468112c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:39.824295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab545017-b163-42fb-b457-032468112c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:39.824822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab545017-b163-42fb-b457-032468112c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:39.844777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 883/1750] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:39.846901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d8d65-e735-452f-841b-cf861ae1ccc4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:39.850931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab545017-b163-42fb-b457-032468112c11 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:39.930028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d8d65-e735-452f-841b-cf861ae1ccc4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:39.950651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e513f0-6952-4664-a30d-816da92cedc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:39.951899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e513f0-6952-4664-a30d-816da92cedc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:39.952420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e513f0-6952-4664-a30d-816da92cedc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:39.952970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e513f0-6952-4664-a30d-816da92cedc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:39.954003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e513f0-6952-4664-a30d-816da92cedc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:39.954370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e513f0-6952-4664-a30d-816da92cedc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:39.956149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e513f0-6952-4664-a30d-816da92cedc7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:40.001215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 868/1751] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:39 2022] POST /identity/v3/projects => generated 379 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:40.002719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab545017-b163-42fb-b457-032468112c11 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:40.011860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 884/1752] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:40.013458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e513f0-6952-4664-a30d-816da92cedc7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:40.048419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e513f0-6952-4664-a30d-816da92cedc7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:40.234911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab545017-b163-42fb-b457-032468112c11 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:40.251539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3227916-b711-479b-8a2e-6f42abb1912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:40.253201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3227916-b711-479b-8a2e-6f42abb1912c admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:40.254555 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3227916-b711-479b-8a2e-6f42abb1912c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:40.255255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3227916-b711-479b-8a2e-6f42abb1912c admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:40.256124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f022a2f-6fa4-4747-a406-514ebe2b2cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:40.257061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3227916-b711-479b-8a2e-6f42abb1912c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:40.259125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f022a2f-6fa4-4747-a406-514ebe2b2cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:40.259782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f022a2f-6fa4-4747-a406-514ebe2b2cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:40.260210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f022a2f-6fa4-4747-a406-514ebe2b2cd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:40.264790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f022a2f-6fa4-4747-a406-514ebe2b2cd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:40.272419 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3227916-b711-479b-8a2e-6f42abb1912c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:40.276874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f022a2f-6fa4-4747-a406-514ebe2b2cd2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:40.336585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:40.357546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 869/1753] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:40 2022] POST /identity/v3/users => generated 290 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:40.362650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3227916-b711-479b-8a2e-6f42abb1912c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:40.364734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:40.389892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 885/1754] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:40 2022] POST /identity/v3/projects => generated 367 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:40.392558 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f022a2f-6fa4-4747-a406-514ebe2b2cd2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:40.411872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3227916-b711-479b-8a2e-6f42abb1912c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:40.770626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa577d9-17ca-4974-a708-7d294e39177b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:40.775055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa577d9-17ca-4974-a708-7d294e39177b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:40.776008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f022a2f-6fa4-4747-a406-514ebe2b2cd2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:40.776884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa577d9-17ca-4974-a708-7d294e39177b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:40.779512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa577d9-17ca-4974-a708-7d294e39177b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:40.781394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa577d9-17ca-4974-a708-7d294e39177b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:40.787467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa577d9-17ca-4974-a708-7d294e39177b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:40.799354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320dfd7-5a4c-4bdb-9566-f8f93942799e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:40.803881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320dfd7-5a4c-4bdb-9566-f8f93942799e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:40.805139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320dfd7-5a4c-4bdb-9566-f8f93942799e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:40.806211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320dfd7-5a4c-4bdb-9566-f8f93942799e admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:40.813166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320dfd7-5a4c-4bdb-9566-f8f93942799e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:40.817196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320dfd7-5a4c-4bdb-9566-f8f93942799e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:40.851585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 870/1755] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:40.857763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a904e-6387-42ef-a79e-9a8eb383359d None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:40.865366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a904e-6387-42ef-a79e-9a8eb383359d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:40.865796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a904e-6387-42ef-a79e-9a8eb383359d None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:40.875964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 871/1756] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 17:07:40 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:07:40.878776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a904e-6387-42ef-a79e-9a8eb383359d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:40.890638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5b0a904e-6387-42ef-a79e-9a8eb383359d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:40.944266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 886/1757] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:40 2022] POST /identity/v3/users => generated 284 bytes in 545 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:40.945573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320dfd7-5a4c-4bdb-9566-f8f93942799e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:41.005700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320dfd7-5a4c-4bdb-9566-f8f93942799e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:41.035983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249bdaa-7c56-4f34-b2e7-42feb6df4d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:41.039261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249bdaa-7c56-4f34-b2e7-42feb6df4d12 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:41.039787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249bdaa-7c56-4f34-b2e7-42feb6df4d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:41.040058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249bdaa-7c56-4f34-b2e7-42feb6df4d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:41.042246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249bdaa-7c56-4f34-b2e7-42feb6df4d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:41.050761 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249bdaa-7c56-4f34-b2e7-42feb6df4d12 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:41.107451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 887/1758] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:41.108467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249bdaa-7c56-4f34-b2e7-42feb6df4d12 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:41.150083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249bdaa-7c56-4f34-b2e7-42feb6df4d12 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:41.170451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acc13a-6ee6-4c98-ac0a-39f08e7cc795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:41.171303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acc13a-6ee6-4c98-ac0a-39f08e7cc795 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:41.171634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acc13a-6ee6-4c98-ac0a-39f08e7cc795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:41.171909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acc13a-6ee6-4c98-ac0a-39f08e7cc795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:41.172579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13acc13a-6ee6-4c98-ac0a-39f08e7cc795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:41.189691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13acc13a-6ee6-4c98-ac0a-39f08e7cc795 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:41.259877 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 888/1759] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:41.270167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e95635-2a3b-491e-9d3d-1ececf23cc8d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:41.279170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e95635-2a3b-491e-9d3d-1ececf23cc8d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:41.279504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e95635-2a3b-491e-9d3d-1ececf23cc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:41.300875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a904e-6387-42ef-a79e-9a8eb383359d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:41.330379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae456e2e-4c45-478f-bfa0-648cd9135494 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:41.332865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae456e2e-4c45-478f-bfa0-648cd9135494 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:41.333154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae456e2e-4c45-478f-bfa0-648cd9135494 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:41.333427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae456e2e-4c45-478f-bfa0-648cd9135494 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:41.339067 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ae456e2e-4c45-478f-bfa0-648cd9135494 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:41.353729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-96e95635-2a3b-491e-9d3d-1ececf23cc8d None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:41.355022 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-96e95635-2a3b-491e-9d3d-1ececf23cc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:41.415686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 889/1760] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 17:07:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:41.416883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e95635-2a3b-491e-9d3d-1ececf23cc8d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:41.493534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e95635-2a3b-491e-9d3d-1ececf23cc8d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:41.527748 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c46b2-0d22-473f-9b8a-7ad85b4e5b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:41.531303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c46b2-0d22-473f-9b8a-7ad85b4e5b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:41.531542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c46b2-0d22-473f-9b8a-7ad85b4e5b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:41.531822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c46b2-0d22-473f-9b8a-7ad85b4e5b1b admin admin] PATH_INFO: `/v3/projects/dc428bb50e164154be533f11afdd07ae/users/0949a0ddad904857a4babd077dace055/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:41.676077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c46b2-0d22-473f-9b8a-7ad85b4e5b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc428bb50e164154be533f11afdd07ae, user_id=0949a0ddad904857a4babd077dace055, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:41.703744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c46b2-0d22-473f-9b8a-7ad85b4e5b1b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:41.771501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae456e2e-4c45-478f-bfa0-648cd9135494 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:41.778535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae456e2e-4c45-478f-bfa0-648cd9135494 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:41.792732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 890/1761] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:41 2022] PUT /identity/v3/projects/dc428bb50e164154be533f11afdd07ae/users/0949a0ddad904857a4babd077dace055/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:41.794404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c46b2-0d22-473f-9b8a-7ad85b4e5b1b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:41.841559 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 872/1762] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 965 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:41.843178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e6b3e-a58e-4bbf-ad3b-4438630b7f95 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:41.849603 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e6b3e-a58e-4bbf-ad3b-4438630b7f95 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:41.850534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e6b3e-a58e-4bbf-ad3b-4438630b7f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:42.091375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c16e6b3e-a58e-4bbf-ad3b-4438630b7f95 None None] MFA Rules not processed for user `0949a0ddad904857a4babd077dace055`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:42.105793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c16e6b3e-a58e-4bbf-ad3b-4438630b7f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:42.197579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:42.255371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c46b2-0d22-473f-9b8a-7ad85b4e5b1b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:42.276584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69003efb-8855-4ea5-8c14-efb3c5ac336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:42.277484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69003efb-8855-4ea5-8c14-efb3c5ac336a admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:42.279696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69003efb-8855-4ea5-8c14-efb3c5ac336a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:42.280282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69003efb-8855-4ea5-8c14-efb3c5ac336a admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:42.284311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69003efb-8855-4ea5-8c14-efb3c5ac336a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:42.286464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69003efb-8855-4ea5-8c14-efb3c5ac336a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:42.337630 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 891/1763] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 540 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:42.341461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69003efb-8855-4ea5-8c14-efb3c5ac336a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:42.376107 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69003efb-8855-4ea5-8c14-efb3c5ac336a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:42.397721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db008dba-c6a4-4672-a7b5-b61e766f08fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:42.399727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db008dba-c6a4-4672-a7b5-b61e766f08fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:42.405535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db008dba-c6a4-4672-a7b5-b61e766f08fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:42.410196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db008dba-c6a4-4672-a7b5-b61e766f08fe admin admin] PATH_INFO: `/v3/projects/230f44c7b87b453ba7211e6aad3925ff/users/d393bfedb7314b6e87903dd4ce124fa3/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:42.450762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 873/1764] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 608 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:42.457005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e6b3e-a58e-4bbf-ad3b-4438630b7f95 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:42.488034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db008dba-c6a4-4672-a7b5-b61e766f08fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=230f44c7b87b453ba7211e6aad3925ff, user_id=d393bfedb7314b6e87903dd4ce124fa3, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:42.499944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db008dba-c6a4-4672-a7b5-b61e766f08fe admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:42.518405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e6b3e-a58e-4bbf-ad3b-4438630b7f95 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:42.554099 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd77cd-1097-49ff-a611-f3168a83001e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:42.556663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd77cd-1097-49ff-a611-f3168a83001e admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:42.561685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd77cd-1097-49ff-a611-f3168a83001e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:42.563066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd77cd-1097-49ff-a611-f3168a83001e admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:42.566315 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd77cd-1097-49ff-a611-f3168a83001e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:42.573774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd77cd-1097-49ff-a611-f3168a83001e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:42.579402 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 892/1765] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:42 2022] PUT /identity/v3/projects/230f44c7b87b453ba7211e6aad3925ff/users/d393bfedb7314b6e87903dd4ce124fa3/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:42.602286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e8bfe-1c90-472b-b4b6-24e4a622f74a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:42.602734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e8bfe-1c90-472b-b4b6-24e4a622f74a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:42.603161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e8bfe-1c90-472b-b4b6-24e4a622f74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:42.723263 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:42.741743 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 874/1766] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:42 2022] POST /identity/v3/projects => generated 379 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:42.748920 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd77cd-1097-49ff-a611-f3168a83001e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:42.758752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9bbd77cd-1097-49ff-a611-f3168a83001e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:42.763905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-053e8bfe-1c90-472b-b4b6-24e4a622f74a None None] MFA Rules not processed for user `d393bfedb7314b6e87903dd4ce124fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:42.765539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-053e8bfe-1c90-472b-b4b6-24e4a622f74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:43.081301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 893/1767] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:43.084357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e8bfe-1c90-472b-b4b6-24e4a622f74a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:43.124753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd77cd-1097-49ff-a611-f3168a83001e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:43.161959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0e6c2-b6a4-48b8-91db-0825b4d2898c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:43.167914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0e6c2-b6a4-48b8-91db-0825b4d2898c service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:43.171012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0e6c2-b6a4-48b8-91db-0825b4d2898c service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:43.171494 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0e6c2-b6a4-48b8-91db-0825b4d2898c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:43.217567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0e6c2-b6a4-48b8-91db-0825b4d2898c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:43.219647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0e6c2-b6a4-48b8-91db-0825b4d2898c service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:43.297084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 875/1768] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:43.301017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2a4e7-3a6b-458c-88d7-de5ea7dc878e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:43.302389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2a4e7-3a6b-458c-88d7-de5ea7dc878e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:43.304352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2a4e7-3a6b-458c-88d7-de5ea7dc878e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:43.411639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-07c2a4e7-3a6b-458c-88d7-de5ea7dc878e None None] MFA Rules not processed for user `bc8c2a5413c943ed94d76eceb2720f17`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:43.421867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-07c2a4e7-3a6b-458c-88d7-de5ea7dc878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:43.462387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e8bfe-1c90-472b-b4b6-24e4a622f74a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:43.498167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ec4b9-84b8-404e-8a46-448b75016506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:43.503943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ec4b9-84b8-404e-8a46-448b75016506 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:43.504349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ec4b9-84b8-404e-8a46-448b75016506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:43.508240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ec4b9-84b8-404e-8a46-448b75016506 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:43.510796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ec4b9-84b8-404e-8a46-448b75016506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:43.514678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ec4b9-84b8-404e-8a46-448b75016506 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:43.688525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 894/1769] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:43 2022] POST /identity/v3/users => generated 296 bytes in 606 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:43.689791 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ec4b9-84b8-404e-8a46-448b75016506 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:43.775200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ec4b9-84b8-404e-8a46-448b75016506 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:43.803921 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31639b-c302-48ba-9684-5bf9cb3ac541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:43.804808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31639b-c302-48ba-9684-5bf9cb3ac541 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:43.805232 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31639b-c302-48ba-9684-5bf9cb3ac541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:43.805794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31639b-c302-48ba-9684-5bf9cb3ac541 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:43.807283 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31639b-c302-48ba-9684-5bf9cb3ac541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:43.823622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 876/1770] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 526 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:43.824680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2a4e7-3a6b-458c-88d7-de5ea7dc878e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:43.836544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31639b-c302-48ba-9684-5bf9cb3ac541 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:43.888437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2a4e7-3a6b-458c-88d7-de5ea7dc878e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:43.929268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1870d1f-e89d-4c2b-8c4b-b060ceda8be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:43.930063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1870d1f-e89d-4c2b-8c4b-b060ceda8be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:43.930415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1870d1f-e89d-4c2b-8c4b-b060ceda8be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:43.930836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1870d1f-e89d-4c2b-8c4b-b060ceda8be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:43.932029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1870d1f-e89d-4c2b-8c4b-b060ceda8be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:43.935181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1870d1f-e89d-4c2b-8c4b-b060ceda8be1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:44.021822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 895/1771] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:43 2022] POST /identity/v3/projects => generated 367 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:44.022026 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 877/1772] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:44.026808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1870d1f-e89d-4c2b-8c4b-b060ceda8be1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:44.027102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31639b-c302-48ba-9684-5bf9cb3ac541 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:44.078564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1870d1f-e89d-4c2b-8c4b-b060ceda8be1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:44.203901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:44.526024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e815cf-a658-4e38-89bb-27d8418b1bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:44.532570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e815cf-a658-4e38-89bb-27d8418b1bba admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:44.537697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e815cf-a658-4e38-89bb-27d8418b1bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:44.540952 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e815cf-a658-4e38-89bb-27d8418b1bba admin admin] PATH_INFO: `/v3/projects/0e5b8b30d6114ffb9fb5240a92bacd79/users/94e8659fbc1c4dcdbf9511004f4d9e01/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:44.542648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31639b-c302-48ba-9684-5bf9cb3ac541 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:44.602693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67485e6-6271-4c8a-86c7-a8d231fe38f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:44.611119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67485e6-6271-4c8a-86c7-a8d231fe38f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:44.611726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67485e6-6271-4c8a-86c7-a8d231fe38f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:44.615237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67485e6-6271-4c8a-86c7-a8d231fe38f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:44.620100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67485e6-6271-4c8a-86c7-a8d231fe38f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:44.633500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67485e6-6271-4c8a-86c7-a8d231fe38f6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:44.652723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e815cf-a658-4e38-89bb-27d8418b1bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5b8b30d6114ffb9fb5240a92bacd79, user_id=94e8659fbc1c4dcdbf9511004f4d9e01, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:44.666054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e815cf-a658-4e38-89bb-27d8418b1bba admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:44.707534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:44.712794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 878/1773] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:44 2022] PUT /identity/v3/projects/0e5b8b30d6114ffb9fb5240a92bacd79/users/94e8659fbc1c4dcdbf9511004f4d9e01/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:44.719137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e815cf-a658-4e38-89bb-27d8418b1bba admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:44.815012 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 896/1774] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:44 2022] POST /identity/v3/users => generated 290 bytes in 781 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:44.815012 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67485e6-6271-4c8a-86c7-a8d231fe38f6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:44.892446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67485e6-6271-4c8a-86c7-a8d231fe38f6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:45.155460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e815cf-a658-4e38-89bb-27d8418b1bba admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:45.194765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb3abb-7766-495f-8b16-36a3dfe809ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:45.196021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb3abb-7766-495f-8b16-36a3dfe809ef admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:45.196525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb3abb-7766-495f-8b16-36a3dfe809ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:45.197017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb3abb-7766-495f-8b16-36a3dfe809ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:45.198228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb3abb-7766-495f-8b16-36a3dfe809ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:45.211551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb3abb-7766-495f-8b16-36a3dfe809ef admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:45.288334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 879/1775] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 570 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:45.290158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb3abb-7766-495f-8b16-36a3dfe809ef admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:45.358740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb3abb-7766-495f-8b16-36a3dfe809ef admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:45.375410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a35a00-806c-467c-84f9-6ad26f7cbca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:45.382390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a35a00-806c-467c-84f9-6ad26f7cbca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:45.382711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a35a00-806c-467c-84f9-6ad26f7cbca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:45.386383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a35a00-806c-467c-84f9-6ad26f7cbca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:45.387597 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a35a00-806c-467c-84f9-6ad26f7cbca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:45.389478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a35a00-806c-467c-84f9-6ad26f7cbca8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:45.423882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a9867-c7ea-46d3-80d9-45f561c4af30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:45.424675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a9867-c7ea-46d3-80d9-45f561c4af30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:45.424988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a9867-c7ea-46d3-80d9-45f561c4af30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:45.425249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a9867-c7ea-46d3-80d9-45f561c4af30 admin admin] PATH_INFO: `/v3/projects/0e5b8b30d6114ffb9fb5240a92bacd79/users/94e8659fbc1c4dcdbf9511004f4d9e01/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:45.440293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a9867-c7ea-46d3-80d9-45f561c4af30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5b8b30d6114ffb9fb5240a92bacd79, user_id=94e8659fbc1c4dcdbf9511004f4d9e01, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:45.452213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a9867-c7ea-46d3-80d9-45f561c4af30 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:45.500999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 880/1776] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:45 2022] PUT /identity/v3/projects/0e5b8b30d6114ffb9fb5240a92bacd79/users/94e8659fbc1c4dcdbf9511004f4d9e01/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:45.531753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245225d-279f-4d28-9544-fe43cecff2c7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:45.532276 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245225d-279f-4d28-9544-fe43cecff2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:45.533246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245225d-279f-4d28-9544-fe43cecff2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:45.611540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 897/1777] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 804 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:45.613449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a35a00-806c-467c-84f9-6ad26f7cbca8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:45.632608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a245225d-279f-4d28-9544-fe43cecff2c7 None None] MFA Rules not processed for user `94e8659fbc1c4dcdbf9511004f4d9e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:45.633955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a245225d-279f-4d28-9544-fe43cecff2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:45.742657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a35a00-806c-467c-84f9-6ad26f7cbca8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:46.020946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 881/1778] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:46.020946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ae7a9-36a1-44c5-800a-6078f0f7fb96 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:46.020946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ae7a9-36a1-44c5-800a-6078f0f7fb96 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:46.020946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ae7a9-36a1-44c5-800a-6078f0f7fb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:46.075360 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-238ae7a9-36a1-44c5-800a-6078f0f7fb96 None None] MFA Rules not processed for user `b34e9318826c49ed891dde1f7782898b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:46.077623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-238ae7a9-36a1-44c5-800a-6078f0f7fb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:46.089106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:46.194094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93ecca-90f2-48ce-83df-1683fc61b2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:46.195872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93ecca-90f2-48ce-83df-1683fc61b2c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:46.197275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93ecca-90f2-48ce-83df-1683fc61b2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:46.198256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93ecca-90f2-48ce-83df-1683fc61b2c8 admin admin] PATH_INFO: `/v3/projects/394c0dcc137c4255821c5c9c86c10b48/users/18e6902db9dd4267bfdf0a100ad089f5/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:46.239060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93ecca-90f2-48ce-83df-1683fc61b2c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=394c0dcc137c4255821c5c9c86c10b48, user_id=18e6902db9dd4267bfdf0a100ad089f5, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:46.261605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93ecca-90f2-48ce-83df-1683fc61b2c8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:46.298106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 898/1779] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:45 2022] PUT /identity/v3/projects/394c0dcc137c4255821c5c9c86c10b48/users/18e6902db9dd4267bfdf0a100ad089f5/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:46.302298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93ecca-90f2-48ce-83df-1683fc61b2c8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:46.304217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9f93ecca-90f2-48ce-83df-1683fc61b2c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:46.392942 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 882/1780] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:46.399642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ae7a9-36a1-44c5-800a-6078f0f7fb96 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:46.513596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ae7a9-36a1-44c5-800a-6078f0f7fb96 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:46.589663 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93ecca-90f2-48ce-83df-1683fc61b2c8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:46.612777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623c969-02ba-4a01-9588-08533f28242c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:46.613781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623c969-02ba-4a01-9588-08533f28242c service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:46.614084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623c969-02ba-4a01-9588-08533f28242c service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:46.614398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623c969-02ba-4a01-9588-08533f28242c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:46.820928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d273c78-5acc-4bfe-a75a-26af560fe3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:46.821772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d273c78-5acc-4bfe-a75a-26af560fe3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:46.824003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d273c78-5acc-4bfe-a75a-26af560fe3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:46.824475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d273c78-5acc-4bfe-a75a-26af560fe3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:46.825538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d273c78-5acc-4bfe-a75a-26af560fe3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:46.828552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d273c78-5acc-4bfe-a75a-26af560fe3b1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:46.852469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:46.899713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 883/1781] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:46.901233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d273c78-5acc-4bfe-a75a-26af560fe3b1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:46.928902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d273c78-5acc-4bfe-a75a-26af560fe3b1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:46.959963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06732c6-2381-4074-8715-f3b8a34be42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:46.960786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06732c6-2381-4074-8715-f3b8a34be42a admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:46.961140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06732c6-2381-4074-8715-f3b8a34be42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:46.961478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06732c6-2381-4074-8715-f3b8a34be42a admin admin] PATH_INFO: `/v3/projects/394c0dcc137c4255821c5c9c86c10b48/users/18e6902db9dd4267bfdf0a100ad089f5/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:46.966059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06732c6-2381-4074-8715-f3b8a34be42a admin admin] RBAC: Authorizing `identity:create_grant(project_id=394c0dcc137c4255821c5c9c86c10b48, user_id=18e6902db9dd4267bfdf0a100ad089f5, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:46.970223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06732c6-2381-4074-8715-f3b8a34be42a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:47.025887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 884/1782] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:46 2022] PUT /identity/v3/projects/394c0dcc137c4255821c5c9c86c10b48/users/18e6902db9dd4267bfdf0a100ad089f5/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:47.032158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00367328-d823-41a5-90a2-f14e55e868d8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:47.032791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00367328-d823-41a5-90a2-f14e55e868d8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:47.033511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00367328-d823-41a5-90a2-f14e55e868d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:47.105189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623c969-02ba-4a01-9588-08533f28242c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:47.116894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623c969-02ba-4a01-9588-08533f28242c service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:47.136087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-00367328-d823-41a5-90a2-f14e55e868d8 None None] MFA Rules not processed for user `18e6902db9dd4267bfdf0a100ad089f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:47.137798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-00367328-d823-41a5-90a2-f14e55e868d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:47.205985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 899/1783] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 907 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:47.454403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 885/1784] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:07:47.470426 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623c969-02ba-4a01-9588-08533f28242c service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:47.492421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623c969-02ba-4a01-9588-08533f28242c service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:47.846638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfda5f2-f9b0-4da8-b762-9eee405ff1c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:47.860015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfda5f2-f9b0-4da8-b762-9eee405ff1c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:47.860392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfda5f2-f9b0-4da8-b762-9eee405ff1c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:47.860593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfda5f2-f9b0-4da8-b762-9eee405ff1c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:47.895415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfda5f2-f9b0-4da8-b762-9eee405ff1c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:47.900372 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfda5f2-f9b0-4da8-b762-9eee405ff1c0 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:47.954766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 900/1785] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:48.153802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00367328-d823-41a5-90a2-f14e55e868d8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:48.165688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00367328-d823-41a5-90a2-f14e55e868d8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:48.185420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ffb17-ef91-4216-b27b-467bc609c838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:48.187248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ffb17-ef91-4216-b27b-467bc609c838 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:48.187550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ffb17-ef91-4216-b27b-467bc609c838 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:48.187824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ffb17-ef91-4216-b27b-467bc609c838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:48.276322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfda5f2-f9b0-4da8-b762-9eee405ff1c0 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:48.312744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:48.473216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfda5f2-f9b0-4da8-b762-9eee405ff1c0 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:48.498118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee488c3-04fb-4d78-ae0d-68283f1d9a1c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94e8659fbc1c4dcdbf9511004f4d9e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e5b8b30d6114ffb9fb5240a92bacd79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:48.498775 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee488c3-04fb-4d78-ae0d-68283f1d9a1c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:48.499095 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee488c3-04fb-4d78-ae0d-68283f1d9a1c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:48.499379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee488c3-04fb-4d78-ae0d-68283f1d9a1c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:48.500080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee488c3-04fb-4d78-ae0d-68283f1d9a1c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:48.500392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee488c3-04fb-4d78-ae0d-68283f1d9a1c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:48.502332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee488c3-04fb-4d78-ae0d-68283f1d9a1c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:48.515690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ffb17-ef91-4216-b27b-467bc609c838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:48.541448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ffb17-ef91-4216-b27b-467bc609c838 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:48.566182 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 901/1786] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:48.570761 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee488c3-04fb-4d78-ae0d-68283f1d9a1c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:48.575790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 886/1787] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:48.595574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee488c3-04fb-4d78-ae0d-68283f1d9a1c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:48.613891 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ab1da-1916-4693-a04d-4bc5195a3940 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94e8659fbc1c4dcdbf9511004f4d9e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e5b8b30d6114ffb9fb5240a92bacd79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:48.619637 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ab1da-1916-4693-a04d-4bc5195a3940 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:48.619909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ab1da-1916-4693-a04d-4bc5195a3940 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:48.622402 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ab1da-1916-4693-a04d-4bc5195a3940 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:48.623228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ab1da-1916-4693-a04d-4bc5195a3940 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:48.625233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ab1da-1916-4693-a04d-4bc5195a3940 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:48.731944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 902/1788] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:48 2022] POST /identity/v3/projects => generated 355 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:48.735667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ffb17-ef91-4216-b27b-467bc609c838 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:48.789234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ffb17-ef91-4216-b27b-467bc609c838 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:49.249490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f20b3-ca04-4257-b990-a15cad7c556c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94e8659fbc1c4dcdbf9511004f4d9e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e5b8b30d6114ffb9fb5240a92bacd79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:49.253550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f20b3-ca04-4257-b990-a15cad7c556c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:49.254079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f20b3-ca04-4257-b990-a15cad7c556c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:49.254593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f20b3-ca04-4257-b990-a15cad7c556c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:49.260699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f20b3-ca04-4257-b990-a15cad7c556c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:49.261224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f20b3-ca04-4257-b990-a15cad7c556c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:49.268495 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f20b3-ca04-4257-b990-a15cad7c556c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:49.334519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 887/1789] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 600 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:49.422741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ab1da-1916-4693-a04d-4bc5195a3940 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:49.442099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ab1da-1916-4693-a04d-4bc5195a3940 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:49.460781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94e8659fbc1c4dcdbf9511004f4d9e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e5b8b30d6114ffb9fb5240a92bacd79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:49.464920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:49.472555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:49.473116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] PATH_INFO: `/v3/projects/98f6d481ea944d0f8f4671aa356823ce` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:49.475628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: Authorizing `identity:delete_project(project_id=98f6d481ea944d0f8f4671aa356823ce)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:49.486403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:49.690770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:49.784552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f6d481ea944d0f8f4671aa356823ce', 'request_id': 'req-3b1fd33c-5ae0-4149-bc3f-361a3a149003'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:49.822136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f6d481ea944d0f8f4671aa356823ce', 'request_id': 'req-3b1fd33c-5ae0-4149-bc3f-361a3a149003'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:49.853186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f6d481ea944d0f8f4671aa356823ce', 'request_id': 'req-3b1fd33c-5ae0-4149-bc3f-361a3a149003'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:49.910383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] The token cache is being invalidate because project 98f6d481ea944d0f8f4671aa356823ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:49.910897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:49.917027 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b1fd33c-5ae0-4149-bc3f-361a3a149003 tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:49.920296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 903/1790] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:49 2022] DELETE /identity/v3/projects/98f6d481ea944d0f8f4671aa356823ce => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:50.546423 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f20b3-ca04-4257-b990-a15cad7c556c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:50.548438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b13f20b3-ca04-4257-b990-a15cad7c556c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:50.734181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f20b3-ca04-4257-b990-a15cad7c556c tempest-VolumeQuotaClassesTest-204753394 tempest-VolumeQuotaClassesTest-204753394-project-admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:50.930390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:50.931598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:50.932484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:50.932885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] PATH_INFO: `/v3/users/0949a0ddad904857a4babd077dace055` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:50.935427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0949a0ddad904857a4babd077dace055)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:50.939557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:50.952187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:51.069408 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0949a0ddad904857a4babd077dace055'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:51.097837 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0949a0ddad904857a4babd077dace055'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:51.125281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0949a0ddad904857a4babd077dace055'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:51.126643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0949a0ddad904857a4babd077dace055'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:51.143652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0949a0ddad904857a4babd077dace055'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:51.158155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 888/1791] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:50 2022] DELETE /identity/v3/users/0949a0ddad904857a4babd077dace055 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:51.165906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810aa08-ea72-4ca0-9838-910a61ee1b70 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:51.166507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810aa08-ea72-4ca0-9838-910a61ee1b70 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:51.166690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810aa08-ea72-4ca0-9838-910a61ee1b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:51.208788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d810aa08-ea72-4ca0-9838-910a61ee1b70 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:51.210768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d810aa08-ea72-4ca0-9838-910a61ee1b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:51.416005 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 904/1792] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:51.448045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:51.451099 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:51.611134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c5193-aedf-436d-9ed1-ad9013cb740d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:51.621373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878e7f1-a558-417f-af33-808d9b76c49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:51.622898 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878e7f1-a558-417f-af33-808d9b76c49f service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:51.623768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878e7f1-a558-417f-af33-808d9b76c49f service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:51.624156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878e7f1-a558-417f-af33-808d9b76c49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:51.650393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878e7f1-a558-417f-af33-808d9b76c49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:51.653885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878e7f1-a558-417f-af33-808d9b76c49f service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:51.671813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 889/1793] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:52.674359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0dda5-b415-4ff3-9e4a-aa31ea11d3ce None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:52.675632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0dda5-b415-4ff3-9e4a-aa31ea11d3ce None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:52.675632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0dda5-b415-4ff3-9e4a-aa31ea11d3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:52.739695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e2b0dda5-b415-4ff3-9e4a-aa31ea11d3ce None None] MFA Rules not processed for user `5feea3b3b7e4401ea7db242cb60fdfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:52.745605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e2b0dda5-b415-4ff3-9e4a-aa31ea11d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:52.826777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878e7f1-a558-417f-af33-808d9b76c49f service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:52.829595 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c878e7f1-a558-417f-af33-808d9b76c49f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:52.873064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878e7f1-a558-417f-af33-808d9b76c49f service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:52.886179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:52.886832 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:52.887336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:52.887786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] PATH_INFO: `/v3/projects/dc428bb50e164154be533f11afdd07ae` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:52.889157 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc428bb50e164154be533f11afdd07ae)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:52.893529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:52.946202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 905/1794] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:53.069772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc428bb50e164154be533f11afdd07ae', 'request_id': 'req-4cece7ef-1d53-4330-945c-d93a00c3719b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:53.082538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc428bb50e164154be533f11afdd07ae', 'request_id': 'req-4cece7ef-1d53-4330-945c-d93a00c3719b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:53.095570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc428bb50e164154be533f11afdd07ae', 'request_id': 'req-4cece7ef-1d53-4330-945c-d93a00c3719b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:53.113093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] The token cache is being invalidate because project dc428bb50e164154be533f11afdd07ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:53.113448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:53.114342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:53.116975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 890/1795] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:52 2022] DELETE /identity/v3/projects/dc428bb50e164154be533f11afdd07ae => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:53.121342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0dda5-b415-4ff3-9e4a-aa31ea11d3ce None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:53.136477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0dda5-b415-4ff3-9e4a-aa31ea11d3ce None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:53.258456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a558fe-2c48-4ca6-b94b-f5e899e66ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:53.259757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a558fe-2c48-4ca6-b94b-f5e899e66ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:53.260254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a558fe-2c48-4ca6-b94b-f5e899e66ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:53.260757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a558fe-2c48-4ca6-b94b-f5e899e66ca2 admin admin] PATH_INFO: `/v3/users/94e8659fbc1c4dcdbf9511004f4d9e01` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:53.262242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a558fe-2c48-4ca6-b94b-f5e899e66ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94e8659fbc1c4dcdbf9511004f4d9e01)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:53.264626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a558fe-2c48-4ca6-b94b-f5e899e66ca2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:53.286069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:53.354350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-73a558fe-2c48-4ca6-b94b-f5e899e66ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94e8659fbc1c4dcdbf9511004f4d9e01'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:53.374174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-73a558fe-2c48-4ca6-b94b-f5e899e66ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94e8659fbc1c4dcdbf9511004f4d9e01'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:53.392908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-73a558fe-2c48-4ca6-b94b-f5e899e66ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94e8659fbc1c4dcdbf9511004f4d9e01'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:53.414693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-73a558fe-2c48-4ca6-b94b-f5e899e66ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94e8659fbc1c4dcdbf9511004f4d9e01'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:53.415492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-73a558fe-2c48-4ca6-b94b-f5e899e66ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94e8659fbc1c4dcdbf9511004f4d9e01'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:53.428154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 906/1796] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:53 2022] DELETE /identity/v3/users/94e8659fbc1c4dcdbf9511004f4d9e01 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:07:54.375262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:54.377444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:54.401058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cece7ef-1d53-4330-945c-d93a00c3719b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:54.416692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:54.514650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:54.516376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:54.517617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:54.517898 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] PATH_INFO: `/v3/projects/0e5b8b30d6114ffb9fb5240a92bacd79` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:54.520183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e5b8b30d6114ffb9fb5240a92bacd79)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:54.523219 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:54.636799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e5b8b30d6114ffb9fb5240a92bacd79', 'request_id': 'req-9e376413-1c3c-4326-bcde-49db6c17b38b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:54.709361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e5b8b30d6114ffb9fb5240a92bacd79', 'request_id': 'req-9e376413-1c3c-4326-bcde-49db6c17b38b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:54.744866 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e5b8b30d6114ffb9fb5240a92bacd79', 'request_id': 'req-9e376413-1c3c-4326-bcde-49db6c17b38b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:54.798827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] The token cache is being invalidate because project 0e5b8b30d6114ffb9fb5240a92bacd79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:07:54.799915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:54.801853 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e376413-1c3c-4326-bcde-49db6c17b38b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:54.810343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 891/1797] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:07:54 2022] DELETE /identity/v3/projects/0e5b8b30d6114ffb9fb5240a92bacd79 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:54.812103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedac980-2ebf-4ce6-a81c-a92dfc9293b5 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:54.813064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedac980-2ebf-4ce6-a81c-a92dfc9293b5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:54.813513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedac980-2ebf-4ce6-a81c-a92dfc9293b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:54.877043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-cedac980-2ebf-4ce6-a81c-a92dfc9293b5 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:54.883734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cedac980-2ebf-4ce6-a81c-a92dfc9293b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:55.095272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 907/1798] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:55.096639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df6ae7-fdb0-4ec0-b5b1-1466ee74b29f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:55.096931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df6ae7-fdb0-4ec0-b5b1-1466ee74b29f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:55.097242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df6ae7-fdb0-4ec0-b5b1-1466ee74b29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:55.142366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e0df6ae7-fdb0-4ec0-b5b1-1466ee74b29f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:55.143633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e0df6ae7-fdb0-4ec0-b5b1-1466ee74b29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:55.202663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 892/1799] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:55.208906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedac980-2ebf-4ce6-a81c-a92dfc9293b5 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:55.224367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedac980-2ebf-4ce6-a81c-a92dfc9293b5 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:55.231266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11550566-ccee-4411-8bc6-d3cd887a4139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:55.232050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550566-ccee-4411-8bc6-d3cd887a4139 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:55.232444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550566-ccee-4411-8bc6-d3cd887a4139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:55.232800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550566-ccee-4411-8bc6-d3cd887a4139 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:55.233585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11550566-ccee-4411-8bc6-d3cd887a4139 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:07:55.234045 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11550566-ccee-4411-8bc6-d3cd887a4139 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:55.235855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11550566-ccee-4411-8bc6-d3cd887a4139 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:55.267873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 908/1800] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:07:55.273484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df6ae7-fdb0-4ec0-b5b1-1466ee74b29f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:55.300684 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df6ae7-fdb0-4ec0-b5b1-1466ee74b29f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:55.320753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814549b-10c4-4362-a23d-8c86afcf2825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:55.321573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814549b-10c4-4362-a23d-8c86afcf2825 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:55.321964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814549b-10c4-4362-a23d-8c86afcf2825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:55.322299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814549b-10c4-4362-a23d-8c86afcf2825 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:55.323046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814549b-10c4-4362-a23d-8c86afcf2825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:55.327976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814549b-10c4-4362-a23d-8c86afcf2825 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:55.385579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 893/1801] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:55 2022] POST /identity/v3/projects => generated 401 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:55.390264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11550566-ccee-4411-8bc6-d3cd887a4139 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:55.406663 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11550566-ccee-4411-8bc6-d3cd887a4139 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:55.556102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd9985-55ca-4340-8fa9-9cfb65cd77f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:55.556736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd9985-55ca-4340-8fa9-9cfb65cd77f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:55.557018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd9985-55ca-4340-8fa9-9cfb65cd77f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:55.557292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd9985-55ca-4340-8fa9-9cfb65cd77f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:55.557941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd9985-55ca-4340-8fa9-9cfb65cd77f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:55.560739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd9985-55ca-4340-8fa9-9cfb65cd77f4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:55.653570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 909/1802] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:55 2022] POST /identity/v3/users => generated 301 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:55.659090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814549b-10c4-4362-a23d-8c86afcf2825 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:55.684739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814549b-10c4-4362-a23d-8c86afcf2825 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:55.702234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39781f28-ff2e-44e2-a135-ca2feb34b4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:55.703015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39781f28-ff2e-44e2-a135-ca2feb34b4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:55.703392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39781f28-ff2e-44e2-a135-ca2feb34b4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:55.703781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39781f28-ff2e-44e2-a135-ca2feb34b4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:55.704607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39781f28-ff2e-44e2-a135-ca2feb34b4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:55.707239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39781f28-ff2e-44e2-a135-ca2feb34b4b9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:55.740115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 894/1803] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:55.744142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd9985-55ca-4340-8fa9-9cfb65cd77f4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:55.753063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:55.766812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd9985-55ca-4340-8fa9-9cfb65cd77f4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:55.786686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced7c1f-524b-4749-9be0-b8dbf31809d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:55.787362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7c1f-524b-4749-9be0-b8dbf31809d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:55.787643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7c1f-524b-4749-9be0-b8dbf31809d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:55.787919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7c1f-524b-4749-9be0-b8dbf31809d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:55.788742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced7c1f-524b-4749-9be0-b8dbf31809d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:55.805320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced7c1f-524b-4749-9be0-b8dbf31809d3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:55.872963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39781f28-ff2e-44e2-a135-ca2feb34b4b9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:55.885810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 910/1804] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:07:55.907708 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39781f28-ff2e-44e2-a135-ca2feb34b4b9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:55.950193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec533563-911e-461a-ba6b-696f4549382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:55.952042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec533563-911e-461a-ba6b-696f4549382e admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:55.952543 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec533563-911e-461a-ba6b-696f4549382e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:55.952791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec533563-911e-461a-ba6b-696f4549382e admin admin] PATH_INFO: `/v3/projects/bc01740615ff4928ae72955b5ccc5b47/users/a8f67ebddbc840878eb0b8f36798d881/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:55.992141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec533563-911e-461a-ba6b-696f4549382e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc01740615ff4928ae72955b5ccc5b47, user_id=a8f67ebddbc840878eb0b8f36798d881, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:56.001148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec533563-911e-461a-ba6b-696f4549382e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:56.024755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 895/1805] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:55 2022] PUT /identity/v3/projects/bc01740615ff4928ae72955b5ccc5b47/users/a8f67ebddbc840878eb0b8f36798d881/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:56.038661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1142f6-f7c0-4881-b70e-25ac838325c6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:56.038980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1142f6-f7c0-4881-b70e-25ac838325c6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:56.039256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1142f6-f7c0-4881-b70e-25ac838325c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:56.174053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-af1142f6-f7c0-4881-b70e-25ac838325c6 None None] MFA Rules not processed for user `a8f67ebddbc840878eb0b8f36798d881`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:56.178064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-af1142f6-f7c0-4881-b70e-25ac838325c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:56.577802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec533563-911e-461a-ba6b-696f4549382e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:56.580782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 911/1806] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:56 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 544 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:56.746118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:56.822333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec533563-911e-461a-ba6b-696f4549382e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:56.840957 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92330d1e-0e3f-4617-8115-0098241ab777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:56.841611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92330d1e-0e3f-4617-8115-0098241ab777 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:56.841892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92330d1e-0e3f-4617-8115-0098241ab777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:56.842171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92330d1e-0e3f-4617-8115-0098241ab777 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:56.842883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92330d1e-0e3f-4617-8115-0098241ab777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:56.844875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92330d1e-0e3f-4617-8115-0098241ab777 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:56.932712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 896/1807] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:07:56 2022] POST /identity/v3/projects => generated 401 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:07:56.937255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1142f6-f7c0-4881-b70e-25ac838325c6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:57.001618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1142f6-f7c0-4881-b70e-25ac838325c6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:57.188818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d84ff-7161-42ef-9045-63d71e977154 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:57.189323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d84ff-7161-42ef-9045-63d71e977154 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:57.189985 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d84ff-7161-42ef-9045-63d71e977154 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:57.192503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 897/1808] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 17:07:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 17:07:57.252252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80acedc-bec6-4348-b67b-71b5bef88590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:57.254319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80acedc-bec6-4348-b67b-71b5bef88590 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:57.254319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80acedc-bec6-4348-b67b-71b5bef88590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:57.254319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80acedc-bec6-4348-b67b-71b5bef88590 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:57.256537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80acedc-bec6-4348-b67b-71b5bef88590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:57.268920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80acedc-bec6-4348-b67b-71b5bef88590 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:57.314148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17f81b-f8f1-485e-80df-d36eff40e796 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:57.314617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17f81b-f8f1-485e-80df-d36eff40e796 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:57.315377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17f81b-f8f1-485e-80df-d36eff40e796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:57.347929 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 912/1809] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:07:56 2022] POST /identity/v3/users => generated 307 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:07:57.352747 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80acedc-bec6-4348-b67b-71b5bef88590 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:57.386298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80acedc-bec6-4348-b67b-71b5bef88590 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:57.388258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7d17f81b-f8f1-485e-80df-d36eff40e796 None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:57.389682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7d17f81b-f8f1-485e-80df-d36eff40e796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:57.402106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7bd52-3550-47f1-b339-1f1152bb3571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:57.404943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7bd52-3550-47f1-b339-1f1152bb3571 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:57.406005 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7bd52-3550-47f1-b339-1f1152bb3571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:57.406481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7bd52-3550-47f1-b339-1f1152bb3571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:57.407458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7bd52-3550-47f1-b339-1f1152bb3571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:57.409352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7bd52-3550-47f1-b339-1f1152bb3571 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:57.424554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 913/1810] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:07:57.429399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7bd52-3550-47f1-b339-1f1152bb3571 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:57.456480 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7bd52-3550-47f1-b339-1f1152bb3571 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:57.466693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164ae2e-bf2e-4aab-aaff-7fb73f3fd12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:57.468018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164ae2e-bf2e-4aab-aaff-7fb73f3fd12d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:57.468629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164ae2e-bf2e-4aab-aaff-7fb73f3fd12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:57.469151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164ae2e-bf2e-4aab-aaff-7fb73f3fd12d admin admin] PATH_INFO: `/v3/projects/d101b35ab6ba492e9f0bea66d5764efe/users/ac0ca637aab84e60b8d630b934473b5d/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:57.504161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164ae2e-bf2e-4aab-aaff-7fb73f3fd12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d101b35ab6ba492e9f0bea66d5764efe, user_id=ac0ca637aab84e60b8d630b934473b5d, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:57.513004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164ae2e-bf2e-4aab-aaff-7fb73f3fd12d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:57.548242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 898/1812] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 17:07:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:57.549497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 914/1812] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:57 2022] PUT /identity/v3/projects/d101b35ab6ba492e9f0bea66d5764efe/users/ac0ca637aab84e60b8d630b934473b5d/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:57.550836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17f81b-f8f1-485e-80df-d36eff40e796 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:57.567461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17f81b-f8f1-485e-80df-d36eff40e796 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:57.698300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c066ed5-72ee-4d40-8a8c-89cbcdc74fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:57.700478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c066ed5-72ee-4d40-8a8c-89cbcdc74fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:57.700921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c066ed5-72ee-4d40-8a8c-89cbcdc74fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:57.701428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c066ed5-72ee-4d40-8a8c-89cbcdc74fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:57.702251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c066ed5-72ee-4d40-8a8c-89cbcdc74fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:57.704560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c066ed5-72ee-4d40-8a8c-89cbcdc74fbb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:57.729623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164ae2e-bf2e-4aab-aaff-7fb73f3fd12d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:57.740940 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 899/1813] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:07:57.750972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164ae2e-bf2e-4aab-aaff-7fb73f3fd12d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:57.768134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbee6c5-6f87-4d5a-87f1-a34f96dc395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:57.769186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbee6c5-6f87-4d5a-87f1-a34f96dc395d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:57.769711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbee6c5-6f87-4d5a-87f1-a34f96dc395d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:57.772293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbee6c5-6f87-4d5a-87f1-a34f96dc395d admin admin] PATH_INFO: `/v3/projects/d101b35ab6ba492e9f0bea66d5764efe/users/ac0ca637aab84e60b8d630b934473b5d/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:57.778824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbee6c5-6f87-4d5a-87f1-a34f96dc395d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d101b35ab6ba492e9f0bea66d5764efe, user_id=ac0ca637aab84e60b8d630b934473b5d, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:57.788532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbee6c5-6f87-4d5a-87f1-a34f96dc395d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:57.813712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 915/1814] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:07:57 2022] PUT /identity/v3/projects/d101b35ab6ba492e9f0bea66d5764efe/users/ac0ca637aab84e60b8d630b934473b5d/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:57.828268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e62400-a1ce-4743-9793-9ffda2080ada None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:57.828556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e62400-a1ce-4743-9793-9ffda2080ada None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:57.828999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e62400-a1ce-4743-9793-9ffda2080ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:57.902073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-11e62400-a1ce-4743-9793-9ffda2080ada None None] MFA Rules not processed for user `ac0ca637aab84e60b8d630b934473b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:57.905048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-11e62400-a1ce-4743-9793-9ffda2080ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:57.986996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:57.991272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbee6c5-6f87-4d5a-87f1-a34f96dc395d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:57.994051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8bbee6c5-6f87-4d5a-87f1-a34f96dc395d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:58.078389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 900/1815] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:57 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:58.080378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:58.090578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e62400-a1ce-4743-9793-9ffda2080ada None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:58.092617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-11e62400-a1ce-4743-9793-9ffda2080ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:58.173324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbee6c5-6f87-4d5a-87f1-a34f96dc395d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:58.178728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d8024-45c0-4f68-9fd0-e76a3be33998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:58.179429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d8024-45c0-4f68-9fd0-e76a3be33998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:58.179712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d8024-45c0-4f68-9fd0-e76a3be33998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:58.179985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d8024-45c0-4f68-9fd0-e76a3be33998 admin admin] PATH_INFO: `/v3/users/d393bfedb7314b6e87903dd4ce124fa3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:58.181272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d8024-45c0-4f68-9fd0-e76a3be33998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d393bfedb7314b6e87903dd4ce124fa3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:58.184458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d8024-45c0-4f68-9fd0-e76a3be33998 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:58.263222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e62400-a1ce-4743-9793-9ffda2080ada None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:58.268023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43488b5-24bc-4c5b-9a5e-2fbdceb3576f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:58.268914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43488b5-24bc-4c5b-9a5e-2fbdceb3576f service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:58.269521 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43488b5-24bc-4c5b-9a5e-2fbdceb3576f service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:58.269812 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43488b5-24bc-4c5b-9a5e-2fbdceb3576f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:58.272823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f57d8024-45c0-4f68-9fd0-e76a3be33998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd393bfedb7314b6e87903dd4ce124fa3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:58.282791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43488b5-24bc-4c5b-9a5e-2fbdceb3576f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:58.289653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f57d8024-45c0-4f68-9fd0-e76a3be33998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd393bfedb7314b6e87903dd4ce124fa3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:58.290053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43488b5-24bc-4c5b-9a5e-2fbdceb3576f service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:58.299181 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f57d8024-45c0-4f68-9fd0-e76a3be33998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd393bfedb7314b6e87903dd4ce124fa3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:58.313447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f57d8024-45c0-4f68-9fd0-e76a3be33998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd393bfedb7314b6e87903dd4ce124fa3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:58.314664 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 901/1816] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:58 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:58.315323 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f57d8024-45c0-4f68-9fd0-e76a3be33998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd393bfedb7314b6e87903dd4ce124fa3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:07:58.317890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43488b5-24bc-4c5b-9a5e-2fbdceb3576f service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:58.322142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b43488b5-24bc-4c5b-9a5e-2fbdceb3576f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:58.330366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 916/1817] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:07:57 2022] DELETE /identity/v3/users/d393bfedb7314b6e87903dd4ce124fa3 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:07:58.336005 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c330cec-6dc3-4737-9838-389d8f7051ef None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:58.336300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c330cec-6dc3-4737-9838-389d8f7051ef None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:58.336644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c330cec-6dc3-4737-9838-389d8f7051ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:58.391491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5c330cec-6dc3-4737-9838-389d8f7051ef None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:07:58.392848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5c330cec-6dc3-4737-9838-389d8f7051ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:58.519209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43488b5-24bc-4c5b-9a5e-2fbdceb3576f service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:58.624149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 917/1818] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:07:58.625105 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c330cec-6dc3-4737-9838-389d8f7051ef None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:58.632590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5c330cec-6dc3-4737-9838-389d8f7051ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:58.724043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:58.745901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c330cec-6dc3-4737-9838-389d8f7051ef None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:58.760426 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b041a-c03b-45a0-9424-5eacb997cb45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:58.761581 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b041a-c03b-45a0-9424-5eacb997cb45 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:58.762226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b041a-c03b-45a0-9424-5eacb997cb45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:58.762897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b041a-c03b-45a0-9424-5eacb997cb45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:58.768408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9a0b041a-c03b-45a0-9424-5eacb997cb45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:58.780950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f38df-40cb-484d-837f-03f03a965198 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:58.782623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f38df-40cb-484d-837f-03f03a965198 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:58.782623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f38df-40cb-484d-837f-03f03a965198 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:58.782865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f38df-40cb-484d-837f-03f03a965198 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:58.786651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-eb7f38df-40cb-484d-837f-03f03a965198 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:58.822832 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f38df-40cb-484d-837f-03f03a965198 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:58.829583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f38df-40cb-484d-837f-03f03a965198 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:58.849575 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:07:58.861045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 902/1819] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:58.861967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f38df-40cb-484d-837f-03f03a965198 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:07:58.879260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-eb7f38df-40cb-484d-837f-03f03a965198 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:07:58.929335 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b041a-c03b-45a0-9424-5eacb997cb45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:58.931362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b041a-c03b-45a0-9424-5eacb997cb45 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:58.947871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 918/1820] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:58 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:07:59.034535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f38df-40cb-484d-837f-03f03a965198 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:07:59.046886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672f104-de01-46a0-991a-ce73ed3e132b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:07:59.047882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f104-de01-46a0-991a-ce73ed3e132b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:07:59.048367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f104-de01-46a0-991a-ce73ed3e132b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:07:59.048863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f104-de01-46a0-991a-ce73ed3e132b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:07:59.062669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672f104-de01-46a0-991a-ce73ed3e132b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:07:59.064500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672f104-de01-46a0-991a-ce73ed3e132b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:07:59.081713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 903/1821] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:00.142553 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b041a-c03b-45a0-9424-5eacb997cb45 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:00.148521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9a0b041a-c03b-45a0-9424-5eacb997cb45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:00.156915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672f104-de01-46a0-991a-ce73ed3e132b service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:00.159252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7672f104-de01-46a0-991a-ce73ed3e132b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:00.180016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672f104-de01-46a0-991a-ce73ed3e132b service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:00.186751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb5a56-34d0-4415-8bae-f2ef9f989942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:00.187585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb5a56-34d0-4415-8bae-f2ef9f989942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:00.188101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb5a56-34d0-4415-8bae-f2ef9f989942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:00.189236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb5a56-34d0-4415-8bae-f2ef9f989942 admin admin] PATH_INFO: `/v3/users/5e462b464e6a4231809d9fd30b44eb6d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:00.191240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b041a-c03b-45a0-9424-5eacb997cb45 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:00.192606 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb5a56-34d0-4415-8bae-f2ef9f989942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e462b464e6a4231809d9fd30b44eb6d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:00.194826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb5a56-34d0-4415-8bae-f2ef9f989942 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:00.203747 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:00.205845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:00.206589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:00.208144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] PATH_INFO: `/v3/projects/230f44c7b87b453ba7211e6aad3925ff` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:00.212638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=230f44c7b87b453ba7211e6aad3925ff)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:00.224898 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:00.351960 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-35bb5a56-34d0-4415-8bae-f2ef9f989942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e462b464e6a4231809d9fd30b44eb6d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.391762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-35bb5a56-34d0-4415-8bae-f2ef9f989942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e462b464e6a4231809d9fd30b44eb6d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.404610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-35bb5a56-34d0-4415-8bae-f2ef9f989942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e462b464e6a4231809d9fd30b44eb6d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.406536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-35bb5a56-34d0-4415-8bae-f2ef9f989942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e462b464e6a4231809d9fd30b44eb6d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.412213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '230f44c7b87b453ba7211e6aad3925ff', 'request_id': 'req-548618c5-dfda-404e-81aa-8d286e21fba3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.421885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-35bb5a56-34d0-4415-8bae-f2ef9f989942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e462b464e6a4231809d9fd30b44eb6d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.425465 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '230f44c7b87b453ba7211e6aad3925ff', 'request_id': 'req-548618c5-dfda-404e-81aa-8d286e21fba3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.438750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '230f44c7b87b453ba7211e6aad3925ff', 'request_id': 'req-548618c5-dfda-404e-81aa-8d286e21fba3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.442708 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 904/1822] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:00 2022] DELETE /identity/v3/users/5e462b464e6a4231809d9fd30b44eb6d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:00.448667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492cae0-94e2-40ff-896a-ba298e2ea791 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:00.450215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492cae0-94e2-40ff-896a-ba298e2ea791 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:00.450215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492cae0-94e2-40ff-896a-ba298e2ea791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:00.479606 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6492cae0-94e2-40ff-896a-ba298e2ea791 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:00.481416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] The token cache is being invalidate because project 230f44c7b87b453ba7211e6aad3925ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:00.481643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.481977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6492cae0-94e2-40ff-896a-ba298e2ea791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:00.482603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.484769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 919/1823] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:00 2022] DELETE /identity/v3/projects/230f44c7b87b453ba7211e6aad3925ff => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:00.489302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:00.491994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:00.589251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548618c5-dfda-404e-81aa-8d286e21fba3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:00.609702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 905/1824] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:00.625714 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492cae0-94e2-40ff-896a-ba298e2ea791 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:00.626255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:00.627190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6492cae0-94e2-40ff-896a-ba298e2ea791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:00.655351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:00.680613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:00.681268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:00.681618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:00.681905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] PATH_INFO: `/v3/users/18e6902db9dd4267bfdf0a100ad089f5` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:00.683131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e6902db9dd4267bfdf0a100ad089f5)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:00.685317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:00.734817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492cae0-94e2-40ff-896a-ba298e2ea791 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:00.739826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84449d-4ef2-47ea-962c-6635b7b5ea4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:00.741150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84449d-4ef2-47ea-962c-6635b7b5ea4d service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:00.741558 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84449d-4ef2-47ea-962c-6635b7b5ea4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:00.742306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84449d-4ef2-47ea-962c-6635b7b5ea4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:00.758570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84449d-4ef2-47ea-962c-6635b7b5ea4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:00.761309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84449d-4ef2-47ea-962c-6635b7b5ea4d service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:00.801288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 906/1825] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:00.805888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8467a3-17ed-45c9-8bda-7e6c1bf7d00b None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:00.806235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8467a3-17ed-45c9-8bda-7e6c1bf7d00b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:00.806507 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8467a3-17ed-45c9-8bda-7e6c1bf7d00b None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:00.808401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e6902db9dd4267bfdf0a100ad089f5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.811265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 907/1826] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 17:08:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:08:00.830694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e6902db9dd4267bfdf0a100ad089f5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.835013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d445d2-dd21-4b48-bde3-5ae5d184f427 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:00.835297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d445d2-dd21-4b48-bde3-5ae5d184f427 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:00.838016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d445d2-dd21-4b48-bde3-5ae5d184f427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:00.872708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e6902db9dd4267bfdf0a100ad089f5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.897157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e6902db9dd4267bfdf0a100ad089f5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.901479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e6902db9dd4267bfdf0a100ad089f5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:00.907544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-95d445d2-dd21-4b48-bde3-5ae5d184f427 None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:00.912234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-95d445d2-dd21-4b48-bde3-5ae5d184f427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:00.930005 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 920/1827] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:00 2022] DELETE /identity/v3/users/18e6902db9dd4267bfdf0a100ad089f5 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:01.144431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 908/1828] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:01.617743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:01.619291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:01.839026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07707565-7606-4fd6-b663-2da0195d8cf3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:01.845748 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d941532-4bcd-4dd0-a5e5-516fdc468adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:01.846394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d941532-4bcd-4dd0-a5e5-516fdc468adc service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:01.846676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d941532-4bcd-4dd0-a5e5-516fdc468adc service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:01.846972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d941532-4bcd-4dd0-a5e5-516fdc468adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:01.857951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d941532-4bcd-4dd0-a5e5-516fdc468adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:01.859713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d941532-4bcd-4dd0-a5e5-516fdc468adc service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:01.873215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 921/1829] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:02.481613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d445d2-dd21-4b48-bde3-5ae5d184f427 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:02.484329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-95d445d2-dd21-4b48-bde3-5ae5d184f427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:02.501355 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d445d2-dd21-4b48-bde3-5ae5d184f427 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:02.689293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:02.690208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:02.690590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:02.691108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] PATH_INFO: `/v3/projects/e4fd50307b4a4e53b628a2e1ec920101` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:02.694972 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fd50307b4a4e53b628a2e1ec920101)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:02.697429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:02.846835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d941532-4bcd-4dd0-a5e5-516fdc468adc service nova] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:02.850715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6d941532-4bcd-4dd0-a5e5-516fdc468adc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:02.897998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d941532-4bcd-4dd0-a5e5-516fdc468adc service nova] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:02.939640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fd50307b4a4e53b628a2e1ec920101', 'request_id': 'req-758c41f1-d8d1-488c-942a-b4f6c8d100c4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:02.950981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:02.978315 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fd50307b4a4e53b628a2e1ec920101', 'request_id': 'req-758c41f1-d8d1-488c-942a-b4f6c8d100c4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:03.011411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fd50307b4a4e53b628a2e1ec920101', 'request_id': 'req-758c41f1-d8d1-488c-942a-b4f6c8d100c4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:03.040051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] The token cache is being invalidate because project e4fd50307b4a4e53b628a2e1ec920101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:03.040365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:03.041762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-758c41f1-d8d1-488c-942a-b4f6c8d100c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:03.050660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 909/1830] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:02 2022] DELETE /identity/v3/projects/e4fd50307b4a4e53b628a2e1ec920101 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:03.055420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b88486-3bc9-4775-b17d-2a4b303b408f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:03.055649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b88486-3bc9-4775-b17d-2a4b303b408f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:03.055930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b88486-3bc9-4775-b17d-2a4b303b408f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:03.067822 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:03.104174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b3b88486-3bc9-4775-b17d-2a4b303b408f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:03.105564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b3b88486-3bc9-4775-b17d-2a4b303b408f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:03.123336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:03.124740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:03.124968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:03.125236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] PATH_INFO: `/v3/projects/394c0dcc137c4255821c5c9c86c10b48` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:03.127520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=394c0dcc137c4255821c5c9c86c10b48)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:03.130397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:03.248943 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 910/1831] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:03.266991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc96315-fbc1-4dad-b15f-e74838479c3b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:03.266991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc96315-fbc1-4dad-b15f-e74838479c3b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:03.266991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc96315-fbc1-4dad-b15f-e74838479c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:03.332387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '394c0dcc137c4255821c5c9c86c10b48', 'request_id': 'req-eb0023b1-e558-4c39-9728-5f86de84dc3c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:03.356119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '394c0dcc137c4255821c5c9c86c10b48', 'request_id': 'req-eb0023b1-e558-4c39-9728-5f86de84dc3c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:03.365984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7dc96315-fbc1-4dad-b15f-e74838479c3b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:03.368759 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7dc96315-fbc1-4dad-b15f-e74838479c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:03.389498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '394c0dcc137c4255821c5c9c86c10b48', 'request_id': 'req-eb0023b1-e558-4c39-9728-5f86de84dc3c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:03.452740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] The token cache is being invalidate because project 394c0dcc137c4255821c5c9c86c10b48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:03.452740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:03.455620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-eb0023b1-e558-4c39-9728-5f86de84dc3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:03.460935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 922/1832] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:02 2022] DELETE /identity/v3/projects/394c0dcc137c4255821c5c9c86c10b48 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:03.468561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4936-8b7d-4b6f-b614-e2c5f150660b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:03.471691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4936-8b7d-4b6f-b614-e2c5f150660b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:03.475282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4936-8b7d-4b6f-b614-e2c5f150660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:03.539736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 911/1833] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:03.541101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc96315-fbc1-4dad-b15f-e74838479c3b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:03.548602 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b07a4936-8b7d-4b6f-b614-e2c5f150660b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:03.550860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b07a4936-8b7d-4b6f-b614-e2c5f150660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:03.562651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc96315-fbc1-4dad-b15f-e74838479c3b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:03.571350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 923/1834] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:03.578240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6fb9d-07e8-414a-9b95-3fbd23c533a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:03.580213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6fb9d-07e8-414a-9b95-3fbd23c533a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:03.581326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6fb9d-07e8-414a-9b95-3fbd23c533a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:03.582165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6fb9d-07e8-414a-9b95-3fbd23c533a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:03.583612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6fb9d-07e8-414a-9b95-3fbd23c533a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:03.584493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6fb9d-07e8-414a-9b95-3fbd23c533a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:03.588949 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6fb9d-07e8-414a-9b95-3fbd23c533a7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:03.596192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a08e3-cb97-447c-b0ad-06cbe86ec2e8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:03.596824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a08e3-cb97-447c-b0ad-06cbe86ec2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:03.597535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a08e3-cb97-447c-b0ad-06cbe86ec2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:03.624513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 912/1835] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:08:03.635145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6fb9d-07e8-414a-9b95-3fbd23c533a7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:03.658648 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6fb9d-07e8-414a-9b95-3fbd23c533a7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:03.666207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c8fb0-ee20-4c1e-8494-aea4350c9c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:03.667475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c8fb0-ee20-4c1e-8494-aea4350c9c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:03.668461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c8fb0-ee20-4c1e-8494-aea4350c9c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:03.669008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c8fb0-ee20-4c1e-8494-aea4350c9c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:03.670517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c8fb0-ee20-4c1e-8494-aea4350c9c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:03.676475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c8fb0-ee20-4c1e-8494-aea4350c9c11 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:03.679523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-458a08e3-cb97-447c-b0ad-06cbe86ec2e8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:03.682535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-458a08e3-cb97-447c-b0ad-06cbe86ec2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:03.707488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 924/1836] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:03.712064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a08e3-cb97-447c-b0ad-06cbe86ec2e8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:03.735371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 913/1837] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:03 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:03.735620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a08e3-cb97-447c-b0ad-06cbe86ec2e8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:03.737494 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c8fb0-ee20-4c1e-8494-aea4350c9c11 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:03.835361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5788b-51db-43ce-aaab-afbb31dba32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:03.837078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5788b-51db-43ce-aaab-afbb31dba32d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:03.837475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5788b-51db-43ce-aaab-afbb31dba32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:03.837848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5788b-51db-43ce-aaab-afbb31dba32d admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:03.839123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5788b-51db-43ce-aaab-afbb31dba32d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:03.839123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5788b-51db-43ce-aaab-afbb31dba32d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:03.841535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5788b-51db-43ce-aaab-afbb31dba32d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:03.843677 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:03.864513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c8fb0-ee20-4c1e-8494-aea4350c9c11 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:03.871744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98a787-4deb-4326-9f15-2c332a486aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:03.872353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98a787-4deb-4326-9f15-2c332a486aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:03.872641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98a787-4deb-4326-9f15-2c332a486aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:03.872917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98a787-4deb-4326-9f15-2c332a486aa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:03.873574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98a787-4deb-4326-9f15-2c332a486aa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:03.875635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 925/1838] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:08:03.876131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98a787-4deb-4326-9f15-2c332a486aa2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:03.876592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5788b-51db-43ce-aaab-afbb31dba32d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:03.896839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5788b-51db-43ce-aaab-afbb31dba32d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:03.905362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2dec8-b5c6-4918-a087-0cf6bf2b9f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:03.906253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2dec8-b5c6-4918-a087-0cf6bf2b9f08 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:03.907311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2dec8-b5c6-4918-a087-0cf6bf2b9f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:03.907781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2dec8-b5c6-4918-a087-0cf6bf2b9f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:03.908966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2dec8-b5c6-4918-a087-0cf6bf2b9f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:03.911361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2dec8-b5c6-4918-a087-0cf6bf2b9f08 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:03.916867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 914/1839] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:03 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:03.918589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98a787-4deb-4326-9f15-2c332a486aa2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:03.949383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98a787-4deb-4326-9f15-2c332a486aa2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:03.967257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3025d-9eee-425a-8de8-3874f6a5356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:03.967916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3025d-9eee-425a-8de8-3874f6a5356e admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:03.968201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3025d-9eee-425a-8de8-3874f6a5356e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:03.968480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3025d-9eee-425a-8de8-3874f6a5356e admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:03.969150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3025d-9eee-425a-8de8-3874f6a5356e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:03.970832 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3025d-9eee-425a-8de8-3874f6a5356e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:03.982876 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 926/1840] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:03 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:03.990500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2dec8-b5c6-4918-a087-0cf6bf2b9f08 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:03.994628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 915/1841] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:03.998903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3025d-9eee-425a-8de8-3874f6a5356e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:04.010270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3025d-9eee-425a-8de8-3874f6a5356e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:04.052454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:04.152171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2dec8-b5c6-4918-a087-0cf6bf2b9f08 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:04.160058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637bd8b4-902a-49fa-ba11-c9f6d58b30c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:04.160815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bd8b4-902a-49fa-ba11-c9f6d58b30c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:04.161100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bd8b4-902a-49fa-ba11-c9f6d58b30c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:04.161377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bd8b4-902a-49fa-ba11-c9f6d58b30c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:04.162107 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637bd8b4-902a-49fa-ba11-c9f6d58b30c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:04.163013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c916f3-38de-4fa7-98ed-8b334b2b3fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:04.163781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637bd8b4-902a-49fa-ba11-c9f6d58b30c2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:04.165046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c916f3-38de-4fa7-98ed-8b334b2b3fef admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:04.165895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c916f3-38de-4fa7-98ed-8b334b2b3fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:04.166917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c916f3-38de-4fa7-98ed-8b334b2b3fef admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:04.168297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c916f3-38de-4fa7-98ed-8b334b2b3fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:04.172934 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c916f3-38de-4fa7-98ed-8b334b2b3fef admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:04.179635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 916/1842] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:04.181824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637bd8b4-902a-49fa-ba11-c9f6d58b30c2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:04.202259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637bd8b4-902a-49fa-ba11-c9f6d58b30c2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:04.221349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed8d1e-2024-4afd-acc8-cbbde0d1e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:04.222725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed8d1e-2024-4afd-acc8-cbbde0d1e259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:04.223505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed8d1e-2024-4afd-acc8-cbbde0d1e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:04.223922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed8d1e-2024-4afd-acc8-cbbde0d1e259 admin admin] PATH_INFO: `/v3/projects/ded031d645c14fbc80a8974d33ab3fd7/users/138eeae792c24240bdcece03ebf04250/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:04.241185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 927/1843] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:03 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:04.248709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c916f3-38de-4fa7-98ed-8b334b2b3fef admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:04.271286 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed8d1e-2024-4afd-acc8-cbbde0d1e259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded031d645c14fbc80a8974d33ab3fd7, user_id=138eeae792c24240bdcece03ebf04250, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:04.283031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c916f3-38de-4fa7-98ed-8b334b2b3fef admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:04.293711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed8d1e-2024-4afd-acc8-cbbde0d1e259 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:04.300620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7573e7-98b2-4bff-8516-1d7dc4746cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:04.303330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7573e7-98b2-4bff-8516-1d7dc4746cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:04.304793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7573e7-98b2-4bff-8516-1d7dc4746cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:04.306380 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7573e7-98b2-4bff-8516-1d7dc4746cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:04.309774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7573e7-98b2-4bff-8516-1d7dc4746cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:04.315860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7573e7-98b2-4bff-8516-1d7dc4746cc3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:04.333778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 917/1844] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:04 2022] PUT /identity/v3/projects/ded031d645c14fbc80a8974d33ab3fd7/users/138eeae792c24240bdcece03ebf04250/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:04.349084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089b5cd-58b7-49d1-9b32-5e9db18af590 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:04.349084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089b5cd-58b7-49d1-9b32-5e9db18af590 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:04.349084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089b5cd-58b7-49d1-9b32-5e9db18af590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:04.351952 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 928/1845] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:04.354077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7573e7-98b2-4bff-8516-1d7dc4746cc3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:04.368658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7573e7-98b2-4bff-8516-1d7dc4746cc3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:04.432946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0089b5cd-58b7-49d1-9b32-5e9db18af590 None None] MFA Rules not processed for user `138eeae792c24240bdcece03ebf04250`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:04.437594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0089b5cd-58b7-49d1-9b32-5e9db18af590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:04.519204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc298c-4c0d-49bd-9b9f-38c7256d21dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:04.520104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc298c-4c0d-49bd-9b9f-38c7256d21dd admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:04.520552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc298c-4c0d-49bd-9b9f-38c7256d21dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:04.521919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc298c-4c0d-49bd-9b9f-38c7256d21dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:04.527756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc298c-4c0d-49bd-9b9f-38c7256d21dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:04.529460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc298c-4c0d-49bd-9b9f-38c7256d21dd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:04.543321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 929/1846] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:04.546662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc298c-4c0d-49bd-9b9f-38c7256d21dd admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:04.563799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc298c-4c0d-49bd-9b9f-38c7256d21dd admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:04.578580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afaaae-f58a-4ec4-a94e-72f1b4f7a295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:04.580356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afaaae-f58a-4ec4-a94e-72f1b4f7a295 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:04.580645 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afaaae-f58a-4ec4-a94e-72f1b4f7a295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:04.580919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afaaae-f58a-4ec4-a94e-72f1b4f7a295 admin admin] PATH_INFO: `/v3/projects/278384ce652c41c4a1d5f5d1e73ef4c3/users/c2db9455496c41f38db3108949fa8beb/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:04.604470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 918/1847] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:04.609199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089b5cd-58b7-49d1-9b32-5e9db18af590 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:04.612163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afaaae-f58a-4ec4-a94e-72f1b4f7a295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=278384ce652c41c4a1d5f5d1e73ef4c3, user_id=c2db9455496c41f38db3108949fa8beb, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:04.621874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afaaae-f58a-4ec4-a94e-72f1b4f7a295 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:04.642649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089b5cd-58b7-49d1-9b32-5e9db18af590 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:04.649828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 930/1848] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:04 2022] PUT /identity/v3/projects/278384ce652c41c4a1d5f5d1e73ef4c3/users/c2db9455496c41f38db3108949fa8beb/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:04.665919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94e9d0-6329-4c1c-a31c-b89c35192132 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:04.666933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94e9d0-6329-4c1c-a31c-b89c35192132 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:04.667254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94e9d0-6329-4c1c-a31c-b89c35192132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:04.732876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:04.758333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3c94e9d0-6329-4c1c-a31c-b89c35192132 None None] MFA Rules not processed for user `c2db9455496c41f38db3108949fa8beb`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:04.762139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3c94e9d0-6329-4c1c-a31c-b89c35192132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:04.844758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81be63b-b544-46da-bb7b-6b56589ee8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:04.847732 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be63b-b544-46da-bb7b-6b56589ee8bb admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:04.847732 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be63b-b544-46da-bb7b-6b56589ee8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:04.850722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be63b-b544-46da-bb7b-6b56589ee8bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:04.854238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81be63b-b544-46da-bb7b-6b56589ee8bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:04.857579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81be63b-b544-46da-bb7b-6b56589ee8bb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:04.901271 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 919/1849] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:04 2022] POST /identity/v3/projects => generated 373 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:04.903213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81be63b-b544-46da-bb7b-6b56589ee8bb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:04.907791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e81be63b-b544-46da-bb7b-6b56589ee8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:04.920271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 931/1850] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:04.921777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94e9d0-6329-4c1c-a31c-b89c35192132 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:04.928780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:04.938787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94e9d0-6329-4c1c-a31c-b89c35192132 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:05.060816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233891f9-eca6-48eb-8748-67f76f4ab99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:05.061647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233891f9-eca6-48eb-8748-67f76f4ab99a admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:05.062104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233891f9-eca6-48eb-8748-67f76f4ab99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:05.062572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233891f9-eca6-48eb-8748-67f76f4ab99a admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:05.063402 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233891f9-eca6-48eb-8748-67f76f4ab99a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:05.065829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233891f9-eca6-48eb-8748-67f76f4ab99a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:05.084029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81be63b-b544-46da-bb7b-6b56589ee8bb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:05.093451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:05.094121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:05.094697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:05.094697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] PATH_INFO: `/v3/users/a8f67ebddbc840878eb0b8f36798d881` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:05.095929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f67ebddbc840878eb0b8f36798d881)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:05.097149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 932/1851] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:04 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:05.098358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:05.102074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233891f9-eca6-48eb-8748-67f76f4ab99a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:05.121041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233891f9-eca6-48eb-8748-67f76f4ab99a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:05.135525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9fa9e-294e-4dc9-b68d-2587cce05720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:05.138116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9fa9e-294e-4dc9-b68d-2587cce05720 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:05.138624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9fa9e-294e-4dc9-b68d-2587cce05720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:05.139217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9fa9e-294e-4dc9-b68d-2587cce05720 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:05.140507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9fa9e-294e-4dc9-b68d-2587cce05720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:05.143348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9fa9e-294e-4dc9-b68d-2587cce05720 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:05.193987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 933/1852] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:05 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:05.195361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9fa9e-294e-4dc9-b68d-2587cce05720 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:05.205659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f67ebddbc840878eb0b8f36798d881'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:05.235398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f67ebddbc840878eb0b8f36798d881'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:05.268370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f67ebddbc840878eb0b8f36798d881'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:05.271268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f67ebddbc840878eb0b8f36798d881'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:05.287136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f67ebddbc840878eb0b8f36798d881'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:05.305615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9fa9e-294e-4dc9-b68d-2587cce05720 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:05.310802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 920/1853] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:04 2022] DELETE /identity/v3/users/a8f67ebddbc840878eb0b8f36798d881 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:05.311482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:05.314023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f4dc8-0008-4131-8ccc-696e11d4fbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:05.314974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f4dc8-0008-4131-8ccc-696e11d4fbbb admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:05.315520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f4dc8-0008-4131-8ccc-696e11d4fbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:05.316041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f4dc8-0008-4131-8ccc-696e11d4fbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:05.316985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f4dc8-0008-4131-8ccc-696e11d4fbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:05.318051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:05.319021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f4dc8-0008-4131-8ccc-696e11d4fbbb admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:05.332423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 934/1854] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:05.333789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3c529-1550-4101-aeb7-1f8448a3e914 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:05.334075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3c529-1550-4101-aeb7-1f8448a3e914 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:05.334545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3c529-1550-4101-aeb7-1f8448a3e914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:05.363275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f5b3c529-1550-4101-aeb7-1f8448a3e914 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:05.366292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f5b3c529-1550-4101-aeb7-1f8448a3e914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:05.368357 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:05.420721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc866e3-87df-4f1c-8df0-358c7a0bedac admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:05.427692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547beb2-2370-4c77-b81c-8b80f4b50cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:05.428620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547beb2-2370-4c77-b81c-8b80f4b50cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:05.429083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547beb2-2370-4c77-b81c-8b80f4b50cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:05.429684 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547beb2-2370-4c77-b81c-8b80f4b50cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:05.430791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547beb2-2370-4c77-b81c-8b80f4b50cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:05.434682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547beb2-2370-4c77-b81c-8b80f4b50cb0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:05.484095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 921/1855] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:05 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:05.485253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547beb2-2370-4c77-b81c-8b80f4b50cb0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:05.487434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5547beb2-2370-4c77-b81c-8b80f4b50cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:05.492823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 935/1856] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:05.494468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3c529-1550-4101-aeb7-1f8448a3e914 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:05.518215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3c529-1550-4101-aeb7-1f8448a3e914 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:05.527753 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0356113-d260-44af-89eb-194c667fa14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:05.528842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356113-d260-44af-89eb-194c667fa14a admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:05.529355 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356113-d260-44af-89eb-194c667fa14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:05.529861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356113-d260-44af-89eb-194c667fa14a admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:05.531103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0356113-d260-44af-89eb-194c667fa14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:05.534433 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547beb2-2370-4c77-b81c-8b80f4b50cb0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:05.535291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0356113-d260-44af-89eb-194c667fa14a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:05.537966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:05.548201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b7f2d-0019-4a80-9288-9c12fff91cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:05.549961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b7f2d-0019-4a80-9288-9c12fff91cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:05.550519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b7f2d-0019-4a80-9288-9c12fff91cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:05.554609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b7f2d-0019-4a80-9288-9c12fff91cc3 admin admin] PATH_INFO: `/v3/projects/aa5500b5e3e140269335f57e8c61c1d8/users/cdf766bd30b84b3786f4704d1e54eeb8/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:05.577106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 936/1857] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:05.583258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0356113-d260-44af-89eb-194c667fa14a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:05.586630 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e0356113-d260-44af-89eb-194c667fa14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:05.660271 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b7f2d-0019-4a80-9288-9c12fff91cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5500b5e3e140269335f57e8c61c1d8, user_id=cdf766bd30b84b3786f4704d1e54eeb8, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:05.678436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b7f2d-0019-4a80-9288-9c12fff91cc3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:05.732373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 922/1858] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:05 2022] PUT /identity/v3/projects/aa5500b5e3e140269335f57e8c61c1d8/users/cdf766bd30b84b3786f4704d1e54eeb8/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:05.734504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b7f2d-0019-4a80-9288-9c12fff91cc3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:05.741659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0356113-d260-44af-89eb-194c667fa14a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:05.955443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb3ead-74c5-4fbf-beab-0aa9822ca227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:05.956419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb3ead-74c5-4fbf-beab-0aa9822ca227 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:05.957272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb3ead-74c5-4fbf-beab-0aa9822ca227 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:05.957566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb3ead-74c5-4fbf-beab-0aa9822ca227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:05.977093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b7f2d-0019-4a80-9288-9c12fff91cc3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:05.990092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb3ead-74c5-4fbf-beab-0aa9822ca227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:05.991522 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb3ead-74c5-4fbf-beab-0aa9822ca227 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:06.005237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab4bb7-3b53-4fd7-89c5-ced3923d6d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:06.005968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab4bb7-3b53-4fd7-89c5-ced3923d6d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:06.006253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab4bb7-3b53-4fd7-89c5-ced3923d6d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:06.006529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab4bb7-3b53-4fd7-89c5-ced3923d6d99 admin admin] PATH_INFO: `/v3/projects/d9c65c038a6946e38691fc3953ed604a/users/e32b53372de3481bb77462706ac5d123/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:06.014049 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 937/1859] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:06.019747 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb3ead-74c5-4fbf-beab-0aa9822ca227 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:06.030671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab4bb7-3b53-4fd7-89c5-ced3923d6d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c65c038a6946e38691fc3953ed604a, user_id=e32b53372de3481bb77462706ac5d123, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:06.033811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab4bb7-3b53-4fd7-89c5-ced3923d6d99 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:06.046928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb3ead-74c5-4fbf-beab-0aa9822ca227 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:06.061134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 923/1860] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:05 2022] PUT /identity/v3/projects/d9c65c038a6946e38691fc3953ed604a/users/e32b53372de3481bb77462706ac5d123/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:06.065807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab4bb7-3b53-4fd7-89c5-ced3923d6d99 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:06.185416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba4704-17d1-4676-961f-1041eeb70943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:06.186328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba4704-17d1-4676-961f-1041eeb70943 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:06.187068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba4704-17d1-4676-961f-1041eeb70943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:06.187510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba4704-17d1-4676-961f-1041eeb70943 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:06.188428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba4704-17d1-4676-961f-1041eeb70943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:06.190667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba4704-17d1-4676-961f-1041eeb70943 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:06.239668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 938/1861] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:06.275937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba4704-17d1-4676-961f-1041eeb70943 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:06.293852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba4704-17d1-4676-961f-1041eeb70943 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:06.309262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747ef2c-5d1e-4d14-8e65-bf552ba1ac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:06.310594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747ef2c-5d1e-4d14-8e65-bf552ba1ac46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:06.311100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747ef2c-5d1e-4d14-8e65-bf552ba1ac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:06.311550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747ef2c-5d1e-4d14-8e65-bf552ba1ac46 admin admin] PATH_INFO: `/v3/projects/aa5500b5e3e140269335f57e8c61c1d8/users/cdf766bd30b84b3786f4704d1e54eeb8/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:06.319056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747ef2c-5d1e-4d14-8e65-bf552ba1ac46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5500b5e3e140269335f57e8c61c1d8, user_id=cdf766bd30b84b3786f4704d1e54eeb8, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:06.322014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747ef2c-5d1e-4d14-8e65-bf552ba1ac46 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:06.339811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab4bb7-3b53-4fd7-89c5-ced3923d6d99 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:06.370224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 939/1862] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:06 2022] PUT /identity/v3/projects/aa5500b5e3e140269335f57e8c61c1d8/users/cdf766bd30b84b3786f4704d1e54eeb8/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:06.372506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872ad81-ea52-4da8-b08d-e4d78e5699c6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:06.375148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872ad81-ea52-4da8-b08d-e4d78e5699c6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:06.375556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872ad81-ea52-4da8-b08d-e4d78e5699c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:06.398645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:06.448200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d872ad81-ea52-4da8-b08d-e4d78e5699c6 None None] MFA Rules not processed for user `cdf766bd30b84b3786f4704d1e54eeb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:06.450653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d872ad81-ea52-4da8-b08d-e4d78e5699c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:06.458508 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6ba0a-934d-4ca7-ba9d-6fa11b8ce9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:06.459415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6ba0a-934d-4ca7-ba9d-6fa11b8ce9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:06.459798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6ba0a-934d-4ca7-ba9d-6fa11b8ce9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:06.460082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6ba0a-934d-4ca7-ba9d-6fa11b8ce9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:06.460789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6ba0a-934d-4ca7-ba9d-6fa11b8ce9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:06.462578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6ba0a-934d-4ca7-ba9d-6fa11b8ce9d7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:06.487363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 924/1863] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:06.488661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6ba0a-934d-4ca7-ba9d-6fa11b8ce9d7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:06.501670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6ba0a-934d-4ca7-ba9d-6fa11b8ce9d7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:06.507497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf6cc1-e5d9-410f-96f2-384681cb781d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:06.508469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf6cc1-e5d9-410f-96f2-384681cb781d admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:06.508757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf6cc1-e5d9-410f-96f2-384681cb781d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:06.509101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf6cc1-e5d9-410f-96f2-384681cb781d admin admin] PATH_INFO: `/v3/projects/d9c65c038a6946e38691fc3953ed604a/users/e32b53372de3481bb77462706ac5d123/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:06.511251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf6cc1-e5d9-410f-96f2-384681cb781d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c65c038a6946e38691fc3953ed604a, user_id=e32b53372de3481bb77462706ac5d123, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:06.514912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf6cc1-e5d9-410f-96f2-384681cb781d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:06.524676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:06.539348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 925/1864] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:06 2022] PUT /identity/v3/projects/d9c65c038a6946e38691fc3953ed604a/users/e32b53372de3481bb77462706ac5d123/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:06.545412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd3339-b6c7-404b-ac4a-1905a5a59c67 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:06.545692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd3339-b6c7-404b-ac4a-1905a5a59c67 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:06.547124 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd3339-b6c7-404b-ac4a-1905a5a59c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:06.597300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 940/1865] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:06.617515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872ad81-ea52-4da8-b08d-e4d78e5699c6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:06.621526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d872ad81-ea52-4da8-b08d-e4d78e5699c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:06.624676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-65cd3339-b6c7-404b-ac4a-1905a5a59c67 None None] MFA Rules not processed for user `e32b53372de3481bb77462706ac5d123`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:06.626233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-65cd3339-b6c7-404b-ac4a-1905a5a59c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:06.747511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 926/1866] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:06.753148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872ad81-ea52-4da8-b08d-e4d78e5699c6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:06.764878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd3339-b6c7-404b-ac4a-1905a5a59c67 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:06.766417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-65cd3339-b6c7-404b-ac4a-1905a5a59c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:06.846496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93e764-eba8-4eba-8144-64259d293d23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:06.847325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93e764-eba8-4eba-8144-64259d293d23 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:06.847753 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93e764-eba8-4eba-8144-64259d293d23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:06.848174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93e764-eba8-4eba-8144-64259d293d23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:06.884485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd3339-b6c7-404b-ac4a-1905a5a59c67 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:06.895773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74699e-2271-4c4e-876b-aeaef8be352f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:06.896574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74699e-2271-4c4e-876b-aeaef8be352f service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:06.896950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74699e-2271-4c4e-876b-aeaef8be352f service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:06.897316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74699e-2271-4c4e-876b-aeaef8be352f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:06.927339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74699e-2271-4c4e-876b-aeaef8be352f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:06.929249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74699e-2271-4c4e-876b-aeaef8be352f service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:06.958516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 927/1867] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:07.060281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93e764-eba8-4eba-8144-64259d293d23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:07.063232 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93e764-eba8-4eba-8144-64259d293d23 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:07.127565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 941/1868] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:07.134570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74699e-2271-4c4e-876b-aeaef8be352f service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:07.144835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74699e-2271-4c4e-876b-aeaef8be352f service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:07.153845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55220a2-f192-4bd6-aaa9-b755f5f5ce4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:07.154915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55220a2-f192-4bd6-aaa9-b755f5f5ce4b service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:07.156000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55220a2-f192-4bd6-aaa9-b755f5f5ce4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:07.156384 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55220a2-f192-4bd6-aaa9-b755f5f5ce4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:07.158588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c55220a2-f192-4bd6-aaa9-b755f5f5ce4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:07.206377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:07.259757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55220a2-f192-4bd6-aaa9-b755f5f5ce4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:07.262320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55220a2-f192-4bd6-aaa9-b755f5f5ce4b service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:07.291723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 928/1869] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:08:07.733316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93e764-eba8-4eba-8144-64259d293d23 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:07.733316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fd93e764-eba8-4eba-8144-64259d293d23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:07.786568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55220a2-f192-4bd6-aaa9-b755f5f5ce4b service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:07.798036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:07.892266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93e764-eba8-4eba-8144-64259d293d23 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:07.926816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:07.927547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:07.927938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:07.932699 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] PATH_INFO: `/v3/projects/bc01740615ff4928ae72955b5ccc5b47` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:07.939101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc01740615ff4928ae72955b5ccc5b47)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:07.947161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:07.954037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55220a2-f192-4bd6-aaa9-b755f5f5ce4b service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:07.960848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:07.961719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:07.962099 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:07.962474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] PATH_INFO: `/v3/users/c2db9455496c41f38db3108949fa8beb` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:07.965048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2db9455496c41f38db3108949fa8beb)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:07.971060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:08.096241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc01740615ff4928ae72955b5ccc5b47', 'request_id': 'req-c9224e2f-95df-4a36-a4f7-a7b83377c182'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.103820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2db9455496c41f38db3108949fa8beb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.106212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc01740615ff4928ae72955b5ccc5b47', 'request_id': 'req-c9224e2f-95df-4a36-a4f7-a7b83377c182'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.117834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc01740615ff4928ae72955b5ccc5b47', 'request_id': 'req-c9224e2f-95df-4a36-a4f7-a7b83377c182'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.122059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2db9455496c41f38db3108949fa8beb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.132771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2db9455496c41f38db3108949fa8beb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.133553 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2db9455496c41f38db3108949fa8beb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.138499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] The token cache is being invalidate because project bc01740615ff4928ae72955b5ccc5b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:08.138900 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.140346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.142941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 942/1870] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:07 2022] DELETE /identity/v3/projects/bc01740615ff4928ae72955b5ccc5b47 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:08.144129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:08.145949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:08.148309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2db9455496c41f38db3108949fa8beb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.160922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 929/1871] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:07 2022] DELETE /identity/v3/users/c2db9455496c41f38db3108949fa8beb => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:08.162452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:08.164832 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:08.258804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9224e2f-95df-4a36-a4f7-a7b83377c182 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:08.302441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:08.309427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d964a-69c3-4dc2-85af-59c275c1b385 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:08.316391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c036d5d-adf2-4453-b0e5-f3eb87125fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:08.317244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c036d5d-adf2-4453-b0e5-f3eb87125fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:08.317754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c036d5d-adf2-4453-b0e5-f3eb87125fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:08.318381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c036d5d-adf2-4453-b0e5-f3eb87125fb4 admin admin] PATH_INFO: `/v3/users/ac0ca637aab84e60b8d630b934473b5d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:08.320673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c036d5d-adf2-4453-b0e5-f3eb87125fb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0ca637aab84e60b8d630b934473b5d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:08.322161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c036d5d-adf2-4453-b0e5-f3eb87125fb4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:08.357199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc28504-2452-4b45-a655-744db9f20890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:08.357860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc28504-2452-4b45-a655-744db9f20890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:08.358139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc28504-2452-4b45-a655-744db9f20890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:08.358413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc28504-2452-4b45-a655-744db9f20890 admin admin] PATH_INFO: `/v3/users/138eeae792c24240bdcece03ebf04250` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:08.359679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc28504-2452-4b45-a655-744db9f20890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=138eeae792c24240bdcece03ebf04250)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:08.361879 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc28504-2452-4b45-a655-744db9f20890 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:08.370021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:08.444919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1c036d5d-adf2-4453-b0e5-f3eb87125fb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0ca637aab84e60b8d630b934473b5d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.446218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5bc28504-2452-4b45-a655-744db9f20890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138eeae792c24240bdcece03ebf04250'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.477463 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5bc28504-2452-4b45-a655-744db9f20890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138eeae792c24240bdcece03ebf04250'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.482270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1c036d5d-adf2-4453-b0e5-f3eb87125fb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0ca637aab84e60b8d630b934473b5d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.496545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5bc28504-2452-4b45-a655-744db9f20890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138eeae792c24240bdcece03ebf04250'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.497095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1c036d5d-adf2-4453-b0e5-f3eb87125fb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0ca637aab84e60b8d630b934473b5d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.498489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1c036d5d-adf2-4453-b0e5-f3eb87125fb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0ca637aab84e60b8d630b934473b5d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.517525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1c036d5d-adf2-4453-b0e5-f3eb87125fb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0ca637aab84e60b8d630b934473b5d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.523211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5bc28504-2452-4b45-a655-744db9f20890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138eeae792c24240bdcece03ebf04250'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.524266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5bc28504-2452-4b45-a655-744db9f20890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138eeae792c24240bdcece03ebf04250'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:08.545311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 943/1872] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:08 2022] DELETE /identity/v3/users/138eeae792c24240bdcece03ebf04250 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:08.545311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e5ee1-ae99-47ff-aec1-48f7679a6e87 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:08.545311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e5ee1-ae99-47ff-aec1-48f7679a6e87 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:08.545311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e5ee1-ae99-47ff-aec1-48f7679a6e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:08.546037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 930/1873] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:08 2022] DELETE /identity/v3/users/ac0ca637aab84e60b8d630b934473b5d => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:08.548352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f825e-ab87-4d0c-8ded-fbaf8f558a48 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:08.549155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f825e-ab87-4d0c-8ded-fbaf8f558a48 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:08.549663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f825e-ab87-4d0c-8ded-fbaf8f558a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:08.576759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d38e5ee1-ae99-47ff-aec1-48f7679a6e87 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:08.578484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d38e5ee1-ae99-47ff-aec1-48f7679a6e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:08.598204 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c52f825e-ab87-4d0c-8ded-fbaf8f558a48 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:08.601306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c52f825e-ab87-4d0c-8ded-fbaf8f558a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:08.677556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 944/1874] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:08.693028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e5ee1-ae99-47ff-aec1-48f7679a6e87 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:08.694996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d38e5ee1-ae99-47ff-aec1-48f7679a6e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:08.708249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 931/1875] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:08.718785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f825e-ab87-4d0c-8ded-fbaf8f558a48 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:08.725480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c52f825e-ab87-4d0c-8ded-fbaf8f558a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:08.878148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e5ee1-ae99-47ff-aec1-48f7679a6e87 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:08.884512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921ad13-d6c6-422c-b5dc-7a1533794a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:08.885662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921ad13-d6c6-422c-b5dc-7a1533794a6c service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:08.886025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921ad13-d6c6-422c-b5dc-7a1533794a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:08.886901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921ad13-d6c6-422c-b5dc-7a1533794a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:08.912427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921ad13-d6c6-422c-b5dc-7a1533794a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:08.914086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921ad13-d6c6-422c-b5dc-7a1533794a6c service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:08.917365 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f825e-ab87-4d0c-8ded-fbaf8f558a48 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:08.923884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a683086-81b6-4b44-94fe-ebc5920ed8c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:08.924626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a683086-81b6-4b44-94fe-ebc5920ed8c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:08.924961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a683086-81b6-4b44-94fe-ebc5920ed8c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:08.925496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a683086-81b6-4b44-94fe-ebc5920ed8c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:08.935267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 945/1876] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:08.943568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a683086-81b6-4b44-94fe-ebc5920ed8c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:08.945512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a683086-81b6-4b44-94fe-ebc5920ed8c5 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:08.964979 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 932/1877] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:09.044699 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921ad13-d6c6-422c-b5dc-7a1533794a6c service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:09.048279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7921ad13-d6c6-422c-b5dc-7a1533794a6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:09.074083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921ad13-d6c6-422c-b5dc-7a1533794a6c service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:09.107676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:09.108969 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:09.109490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:09.109850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] PATH_INFO: `/v3/projects/d101b35ab6ba492e9f0bea66d5764efe` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:09.111594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d101b35ab6ba492e9f0bea66d5764efe)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:09.113798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:09.152734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:09.196955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd101b35ab6ba492e9f0bea66d5764efe', 'request_id': 'req-86f19f6f-2e4b-49c4-bda0-122a687b87d2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.207110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd101b35ab6ba492e9f0bea66d5764efe', 'request_id': 'req-86f19f6f-2e4b-49c4-bda0-122a687b87d2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.219712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd101b35ab6ba492e9f0bea66d5764efe', 'request_id': 'req-86f19f6f-2e4b-49c4-bda0-122a687b87d2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.239251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] The token cache is being invalidate because project d101b35ab6ba492e9f0bea66d5764efe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:09.239523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.240517 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-86f19f6f-2e4b-49c4-bda0-122a687b87d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.243022 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 946/1878] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:09 2022] DELETE /identity/v3/projects/d101b35ab6ba492e9f0bea66d5764efe => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:09.250094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9586190-170d-45f6-9416-b3ee670eb694 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:09.250425 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9586190-170d-45f6-9416-b3ee670eb694 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:09.250698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9586190-170d-45f6-9416-b3ee670eb694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:09.310104 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f9586190-170d-45f6-9416-b3ee670eb694 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:09.320749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f9586190-170d-45f6-9416-b3ee670eb694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:09.350163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:09.431169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 933/1879] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:09.436298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072ed6a-003f-470e-bee8-f5c7906190ee None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:09.436585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072ed6a-003f-470e-bee8-f5c7906190ee None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:09.436862 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072ed6a-003f-470e-bee8-f5c7906190ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:09.460094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f072ed6a-003f-470e-bee8-f5c7906190ee None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:09.461439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f072ed6a-003f-470e-bee8-f5c7906190ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:09.469767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 947/1880] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:09.474890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9586190-170d-45f6-9416-b3ee670eb694 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:09.488299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9586190-170d-45f6-9416-b3ee670eb694 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:09.495085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34799bcd-26a9-4a85-b7f2-c839e6475c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:09.496500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34799bcd-26a9-4a85-b7f2-c839e6475c76 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:09.497054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34799bcd-26a9-4a85-b7f2-c839e6475c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:09.497441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34799bcd-26a9-4a85-b7f2-c839e6475c76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:09.498339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34799bcd-26a9-4a85-b7f2-c839e6475c76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:09.498706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34799bcd-26a9-4a85-b7f2-c839e6475c76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:09.500636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34799bcd-26a9-4a85-b7f2-c839e6475c76 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:09.526989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 934/1881] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:08:09.532467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072ed6a-003f-470e-bee8-f5c7906190ee None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:09.546808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072ed6a-003f-470e-bee8-f5c7906190ee None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:09.555662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170d624-5573-4baf-ae3a-27bde2275fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:09.557289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170d624-5573-4baf-ae3a-27bde2275fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:09.558090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170d624-5573-4baf-ae3a-27bde2275fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:09.558309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170d624-5573-4baf-ae3a-27bde2275fe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:09.559503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170d624-5573-4baf-ae3a-27bde2275fe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:09.563305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170d624-5573-4baf-ae3a-27bde2275fe0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:09.611928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 948/1882] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:09 2022] POST /identity/v3/projects => generated 387 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:09.618614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34799bcd-26a9-4a85-b7f2-c839e6475c76 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:09.637391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34799bcd-26a9-4a85-b7f2-c839e6475c76 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:09.727750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415a158-c85e-4fce-8134-4f439c6d541b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:09.728644 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415a158-c85e-4fce-8134-4f439c6d541b admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:09.729026 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415a158-c85e-4fce-8134-4f439c6d541b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:09.729362 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415a158-c85e-4fce-8134-4f439c6d541b admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:09.731063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415a158-c85e-4fce-8134-4f439c6d541b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:09.733667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415a158-c85e-4fce-8134-4f439c6d541b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:09.776518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170d624-5573-4baf-ae3a-27bde2275fe0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:09.779699 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6170d624-5573-4baf-ae3a-27bde2275fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:09.795741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 935/1883] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:09 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:09.797577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415a158-c85e-4fce-8134-4f439c6d541b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:09.803148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8415a158-c85e-4fce-8134-4f439c6d541b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:09.826266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170d624-5573-4baf-ae3a-27bde2275fe0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:09.826502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415a158-c85e-4fce-8134-4f439c6d541b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:09.835951 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:09.836653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:09.836933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:09.837204 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] PATH_INFO: `/v3/projects/278384ce652c41c4a1d5f5d1e73ef4c3` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:09.838014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:09.839159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=278384ce652c41c4a1d5f5d1e73ef4c3)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:09.840034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:09.842293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:09.846358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:09.847080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] PATH_INFO: `/v3/projects/ded031d645c14fbc80a8974d33ab3fd7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:09.852023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded031d645c14fbc80a8974d33ab3fd7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:09.855886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:09.919925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '278384ce652c41c4a1d5f5d1e73ef4c3', 'request_id': 'req-76abfe4a-fc05-41a4-a16d-82cbae420342'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.929814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '278384ce652c41c4a1d5f5d1e73ef4c3', 'request_id': 'req-76abfe4a-fc05-41a4-a16d-82cbae420342'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.934756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded031d645c14fbc80a8974d33ab3fd7', 'request_id': 'req-08717b75-e0ee-4a68-98c7-e0d529a20372'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.941769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '278384ce652c41c4a1d5f5d1e73ef4c3', 'request_id': 'req-76abfe4a-fc05-41a4-a16d-82cbae420342'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.942204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded031d645c14fbc80a8974d33ab3fd7', 'request_id': 'req-08717b75-e0ee-4a68-98c7-e0d529a20372'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.950031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded031d645c14fbc80a8974d33ab3fd7', 'request_id': 'req-08717b75-e0ee-4a68-98c7-e0d529a20372'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.965456 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] The token cache is being invalidate because project 278384ce652c41c4a1d5f5d1e73ef4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:09.965970 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.967233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.968306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] The token cache is being invalidate because project ded031d645c14fbc80a8974d33ab3fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:09.968769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.969603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:09.970252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 936/1884] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:09 2022] DELETE /identity/v3/projects/278384ce652c41c4a1d5f5d1e73ef4c3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:09.974274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:09.974274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:09.975224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 949/1885] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:09 2022] DELETE /identity/v3/projects/ded031d645c14fbc80a8974d33ab3fd7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:09.976521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:09.978218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:10.006828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:10.066976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08717b75-e0ee-4a68-98c7-e0d529a20372 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:10.071801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:10.072531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:10.072816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:10.073092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] PATH_INFO: `/v3/users/e32b53372de3481bb77462706ac5d123` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:10.074334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32b53372de3481bb77462706ac5d123)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:10.076310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:10.081910 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abfe4a-fc05-41a4-a16d-82cbae420342 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:10.082594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:10.090846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0970342-ddc4-414c-8290-2f916b96056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:10.091926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0970342-ddc4-414c-8290-2f916b96056a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:10.092505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0970342-ddc4-414c-8290-2f916b96056a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:10.092929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0970342-ddc4-414c-8290-2f916b96056a admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:10.094098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0970342-ddc4-414c-8290-2f916b96056a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:10.097259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0970342-ddc4-414c-8290-2f916b96056a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:10.113045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 937/1886] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:10.113994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0970342-ddc4-414c-8290-2f916b96056a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:10.115723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a0970342-ddc4-414c-8290-2f916b96056a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:10.129266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0970342-ddc4-414c-8290-2f916b96056a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:10.134084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:10.134720 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:10.135023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:10.135287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] PATH_INFO: `/v3/users/cdf766bd30b84b3786f4704d1e54eeb8` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:10.136376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32b53372de3481bb77462706ac5d123'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:10.137147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf766bd30b84b3786f4704d1e54eeb8)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:10.138500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:10.153252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32b53372de3481bb77462706ac5d123'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:10.170003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32b53372de3481bb77462706ac5d123'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:10.180956 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32b53372de3481bb77462706ac5d123'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:10.182115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32b53372de3481bb77462706ac5d123'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:10.191946 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 950/1887] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:09 2022] DELETE /identity/v3/users/e32b53372de3481bb77462706ac5d123 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:10.193210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:10.195032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:10.204838 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf766bd30b84b3786f4704d1e54eeb8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:10.231377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf766bd30b84b3786f4704d1e54eeb8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:10.247480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf766bd30b84b3786f4704d1e54eeb8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:10.248774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf766bd30b84b3786f4704d1e54eeb8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:10.264543 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf766bd30b84b3786f4704d1e54eeb8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:10.285446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 938/1888] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:10 2022] DELETE /identity/v3/users/cdf766bd30b84b3786f4704d1e54eeb8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:10.315077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd8b8c-4c70-48a4-b8a5-d343f12ff1c2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:10.492212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ecf5e-f8ac-448d-9b37-78439905d381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:10.492925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ecf5e-f8ac-448d-9b37-78439905d381 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:10.493350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ecf5e-f8ac-448d-9b37-78439905d381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:10.493779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ecf5e-f8ac-448d-9b37-78439905d381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:10.494815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ecf5e-f8ac-448d-9b37-78439905d381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:10.497523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ecf5e-f8ac-448d-9b37-78439905d381 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:10.512894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 951/1889] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:10.518449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:10.520815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:10.549303 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d80493-e673-4e7e-86aa-53dd116b48af admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:10.558796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae74e7-7bad-48aa-9255-a52e3f028d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:10.559517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae74e7-7bad-48aa-9255-a52e3f028d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:10.559995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae74e7-7bad-48aa-9255-a52e3f028d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:10.560777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae74e7-7bad-48aa-9255-a52e3f028d43 admin admin] PATH_INFO: `/v3/projects/39a091e948b64d4ea93bf2075f7c65ba/users/45890d0529dd4f8586c288d5713f4bdc/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:10.593825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae74e7-7bad-48aa-9255-a52e3f028d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a091e948b64d4ea93bf2075f7c65ba, user_id=45890d0529dd4f8586c288d5713f4bdc, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:10.597683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae74e7-7bad-48aa-9255-a52e3f028d43 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:10.624923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 939/1890] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:10 2022] PUT /identity/v3/projects/39a091e948b64d4ea93bf2075f7c65ba/users/45890d0529dd4f8586c288d5713f4bdc/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:10.644780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244a26e-7a13-42ec-93d1-d1329f0e763b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:10.645076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244a26e-7a13-42ec-93d1-d1329f0e763b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:10.645359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244a26e-7a13-42ec-93d1-d1329f0e763b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:10.680212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e244a26e-7a13-42ec-93d1-d1329f0e763b None None] MFA Rules not processed for user `45890d0529dd4f8586c288d5713f4bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:10.681680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e244a26e-7a13-42ec-93d1-d1329f0e763b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:10.737573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:10.810451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae74e7-7bad-48aa-9255-a52e3f028d43 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:10.813479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-dcae74e7-7bad-48aa-9255-a52e3f028d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:10.844265 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 952/1891] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:10.855262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244a26e-7a13-42ec-93d1-d1329f0e763b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:10.944320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae74e7-7bad-48aa-9255-a52e3f028d43 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:10.950813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:10.951700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:10.952946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:10.953209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] PATH_INFO: `/v3/projects/d9c65c038a6946e38691fc3953ed604a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:10.954446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9c65c038a6946e38691fc3953ed604a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:10.956834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:10.964752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:10.982790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244a26e-7a13-42ec-93d1-d1329f0e763b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:10.991730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b5d70-0c84-42ec-b513-6bbc77a53a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:10.992281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b5d70-0c84-42ec-b513-6bbc77a53a76 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:10.992932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b5d70-0c84-42ec-b513-6bbc77a53a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:10.993348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b5d70-0c84-42ec-b513-6bbc77a53a76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:10.993910 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b5d70-0c84-42ec-b513-6bbc77a53a76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:10.996215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b5d70-0c84-42ec-b513-6bbc77a53a76 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:11.020028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9c65c038a6946e38691fc3953ed604a', 'request_id': 'req-1b63367d-2b17-4386-861c-c7acc5e9b04b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:11.031082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9c65c038a6946e38691fc3953ed604a', 'request_id': 'req-1b63367d-2b17-4386-861c-c7acc5e9b04b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:11.039370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 953/1892] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:10 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:11.039971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9c65c038a6946e38691fc3953ed604a', 'request_id': 'req-1b63367d-2b17-4386-861c-c7acc5e9b04b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:11.045660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b5d70-0c84-42ec-b513-6bbc77a53a76 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:11.061530 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] The token cache is being invalidate because project d9c65c038a6946e38691fc3953ed604a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:11.061935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:11.064884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1b63367d-2b17-4386-861c-c7acc5e9b04b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:11.068173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 940/1893] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:10 2022] DELETE /identity/v3/projects/d9c65c038a6946e38691fc3953ed604a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:11.072420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84028dc6-8eed-4863-bec1-6d58ab62998c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.072758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84028dc6-8eed-4863-bec1-6d58ab62998c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.073254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84028dc6-8eed-4863-bec1-6d58ab62998c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.112759 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-84028dc6-8eed-4863-bec1-6d58ab62998c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:11.115090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-84028dc6-8eed-4863-bec1-6d58ab62998c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:11.165471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b5d70-0c84-42ec-b513-6bbc77a53a76 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:11.172066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563bdca-0d8d-450a-9be4-f56318f4f702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:11.172849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563bdca-0d8d-450a-9be4-f56318f4f702 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.173231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563bdca-0d8d-450a-9be4-f56318f4f702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.173577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563bdca-0d8d-450a-9be4-f56318f4f702 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.174607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563bdca-0d8d-450a-9be4-f56318f4f702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:11.179415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563bdca-0d8d-450a-9be4-f56318f4f702 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:11.208649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 954/1894] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:11 2022] POST /identity/v3/users => generated 300 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:11.209639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563bdca-0d8d-450a-9be4-f56318f4f702 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:11.211356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2563bdca-0d8d-450a-9be4-f56318f4f702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:11.225861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563bdca-0d8d-450a-9be4-f56318f4f702 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:11.232189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:11.232844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.233128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.233418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] PATH_INFO: `/v3/projects/aa5500b5e3e140269335f57e8c61c1d8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.234601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa5500b5e3e140269335f57e8c61c1d8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:11.239194 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:11.239379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 941/1895] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:11.239464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84028dc6-8eed-4863-bec1-6d58ab62998c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:11.242174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-84028dc6-8eed-4863-bec1-6d58ab62998c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:11.267507 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84028dc6-8eed-4863-bec1-6d58ab62998c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:11.277922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64398224-73d6-4880-b290-84427e953c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:11.278373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64398224-73d6-4880-b290-84427e953c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.278983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64398224-73d6-4880-b290-84427e953c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.279418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:11.279600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64398224-73d6-4880-b290-84427e953c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.280706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64398224-73d6-4880-b290-84427e953c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:11.284545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64398224-73d6-4880-b290-84427e953c94 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:11.309101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa5500b5e3e140269335f57e8c61c1d8', 'request_id': 'req-df1bf11e-b01e-4ffc-9e11-7fc409306b67'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:11.314163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 942/1896] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:11.316270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b19a1a-fa31-4f03-8841-e5d817bae906 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.316771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b19a1a-fa31-4f03-8841-e5d817bae906 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.317403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b19a1a-fa31-4f03-8841-e5d817bae906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.324919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa5500b5e3e140269335f57e8c61c1d8', 'request_id': 'req-df1bf11e-b01e-4ffc-9e11-7fc409306b67'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:11.348795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa5500b5e3e140269335f57e8c61c1d8', 'request_id': 'req-df1bf11e-b01e-4ffc-9e11-7fc409306b67'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:11.352143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f8b19a1a-fa31-4f03-8841-e5d817bae906 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:11.353393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f8b19a1a-fa31-4f03-8841-e5d817bae906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:11.375391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] The token cache is being invalidate because project aa5500b5e3e140269335f57e8c61c1d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:11.375579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:11.376334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:11.378213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 955/1897] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:11 2022] DELETE /identity/v3/projects/aa5500b5e3e140269335f57e8c61c1d8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:11.379140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:11.380529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:11.485614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bf11e-b01e-4ffc-9e11-7fc409306b67 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:11.487049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 943/1898] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:11.494970 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1071-98e9-4b1b-940a-b30b9301d8d8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.495624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1071-98e9-4b1b-940a-b30b9301d8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.495887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131a3ad-ebcf-4290-90de-a95973e0c563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:11.496327 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1071-98e9-4b1b-940a-b30b9301d8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.496774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131a3ad-ebcf-4290-90de-a95973e0c563 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.497064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131a3ad-ebcf-4290-90de-a95973e0c563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.497343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131a3ad-ebcf-4290-90de-a95973e0c563 admin admin] PATH_INFO: `/v3/projects/15a416986eec4ed58986ccd74ef31fff/users/be45d39363a04b3e8da509a307ce6927/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.517573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131a3ad-ebcf-4290-90de-a95973e0c563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a416986eec4ed58986ccd74ef31fff, user_id=be45d39363a04b3e8da509a307ce6927, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:11.522187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131a3ad-ebcf-4290-90de-a95973e0c563 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:11.541227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e8fe1071-98e9-4b1b-940a-b30b9301d8d8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:11.541659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e8fe1071-98e9-4b1b-940a-b30b9301d8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:11.548692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 956/1899] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:11 2022] PUT /identity/v3/projects/15a416986eec4ed58986ccd74ef31fff/users/be45d39363a04b3e8da509a307ce6927/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:11.550656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131a3ad-ebcf-4290-90de-a95973e0c563 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:11.582932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:11.656771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131a3ad-ebcf-4290-90de-a95973e0c563 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:11.662885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b211f3-3816-4d1c-a909-4e0d73a52279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:11.663725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b211f3-3816-4d1c-a909-4e0d73a52279 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.663885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b211f3-3816-4d1c-a909-4e0d73a52279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.664244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b211f3-3816-4d1c-a909-4e0d73a52279 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.664984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b211f3-3816-4d1c-a909-4e0d73a52279 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:11.665349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b211f3-3816-4d1c-a909-4e0d73a52279 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:11.668347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b211f3-3816-4d1c-a909-4e0d73a52279 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:11.668574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 944/1900] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:11.668719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1071-98e9-4b1b-940a-b30b9301d8d8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:11.681313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1071-98e9-4b1b-940a-b30b9301d8d8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:11.687319 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d22c16-0186-46e0-afe2-a85a752a3af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:11.687319 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d22c16-0186-46e0-afe2-a85a752a3af2 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.687656 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d22c16-0186-46e0-afe2-a85a752a3af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.687895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d22c16-0186-46e0-afe2-a85a752a3af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.688525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d22c16-0186-46e0-afe2-a85a752a3af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:11.690165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d22c16-0186-46e0-afe2-a85a752a3af2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:11.698809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 957/1901] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:08:11.701807 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6362-0573-4aae-bdac-997752c75135 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.702427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6362-0573-4aae-bdac-997752c75135 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.702795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6362-0573-4aae-bdac-997752c75135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.710057 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 945/1902] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:11.710057 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d22c16-0186-46e0-afe2-a85a752a3af2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:11.721125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d22c16-0186-46e0-afe2-a85a752a3af2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:11.728809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6538ec-029f-46cd-84b8-083d9fd42700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:11.730479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6538ec-029f-46cd-84b8-083d9fd42700 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.731016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6538ec-029f-46cd-84b8-083d9fd42700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.731481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6538ec-029f-46cd-84b8-083d9fd42700 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.732615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6538ec-029f-46cd-84b8-083d9fd42700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:11.734673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6538ec-029f-46cd-84b8-083d9fd42700 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:11.744187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-06df6362-0573-4aae-bdac-997752c75135 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:11.745501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-06df6362-0573-4aae-bdac-997752c75135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:11.776759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 958/1903] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:11.776759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df6362-0573-4aae-bdac-997752c75135 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:11.777272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 946/1904] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:11 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:11.777272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6538ec-029f-46cd-84b8-083d9fd42700 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:11.786820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df6362-0573-4aae-bdac-997752c75135 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:11.827823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:11.869595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3484003-e5de-4664-ae28-aee300aa0645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:11.870620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484003-e5de-4664-ae28-aee300aa0645 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.871202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484003-e5de-4664-ae28-aee300aa0645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.871621 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484003-e5de-4664-ae28-aee300aa0645 admin admin] PATH_INFO: `/v3/projects/15a416986eec4ed58986ccd74ef31fff/users/be45d39363a04b3e8da509a307ce6927/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.874239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6538ec-029f-46cd-84b8-083d9fd42700 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:11.874500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3484003-e5de-4664-ae28-aee300aa0645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a416986eec4ed58986ccd74ef31fff, user_id=be45d39363a04b3e8da509a307ce6927, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:11.878568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3484003-e5de-4664-ae28-aee300aa0645 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:11.881472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd68a2e-8a87-476d-a461-ff677d87fcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:11.882504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68a2e-8a87-476d-a461-ff677d87fcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.883011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68a2e-8a87-476d-a461-ff677d87fcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.883494 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68a2e-8a87-476d-a461-ff677d87fcc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:11.884723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd68a2e-8a87-476d-a461-ff677d87fcc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:11.885938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd68a2e-8a87-476d-a461-ff677d87fcc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:11.888660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd68a2e-8a87-476d-a461-ff677d87fcc3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:11.895064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 959/1905] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:11 2022] PUT /identity/v3/projects/15a416986eec4ed58986ccd74ef31fff/users/be45d39363a04b3e8da509a307ce6927/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:11.900729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3484003-e5de-4664-ae28-aee300aa0645 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:11.912047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 947/1906] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:08:11.914794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3924e2-f663-4669-805b-bff6772fd804 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:11.915328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3924e2-f663-4669-805b-bff6772fd804 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:11.915820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3924e2-f663-4669-805b-bff6772fd804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:12.001803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7a3924e2-f663-4669-805b-bff6772fd804 None None] MFA Rules not processed for user `be45d39363a04b3e8da509a307ce6927`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:12.005308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7a3924e2-f663-4669-805b-bff6772fd804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:12.039571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3484003-e5de-4664-ae28-aee300aa0645 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:12.051619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a82c89-3cfb-4d73-a934-bbad6c3831ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:12.052337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82c89-3cfb-4d73-a934-bbad6c3831ff admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:12.052835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82c89-3cfb-4d73-a934-bbad6c3831ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:12.053181 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82c89-3cfb-4d73-a934-bbad6c3831ff admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:12.054029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a82c89-3cfb-4d73-a934-bbad6c3831ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:12.058837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a82c89-3cfb-4d73-a934-bbad6c3831ff admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:12.103457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 960/1907] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:11 2022] POST /identity/v3/users => generated 287 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:12.104420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a82c89-3cfb-4d73-a934-bbad6c3831ff admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:12.118025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a82c89-3cfb-4d73-a934-bbad6c3831ff admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:12.126468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d3e99-d39e-4e74-8b0a-06e6fb704e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:12.127211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d3e99-d39e-4e74-8b0a-06e6fb704e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:12.127502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d3e99-d39e-4e74-8b0a-06e6fb704e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:12.127779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d3e99-d39e-4e74-8b0a-06e6fb704e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:12.128666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d3e99-d39e-4e74-8b0a-06e6fb704e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:12.130519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d3e99-d39e-4e74-8b0a-06e6fb704e35 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:12.148012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 948/1908] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:12.148867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3924e2-f663-4669-805b-bff6772fd804 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:12.165347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3924e2-f663-4669-805b-bff6772fd804 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:12.176274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46477f-5179-4723-959d-e7d16b427ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:12.177388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46477f-5179-4723-959d-e7d16b427ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:12.177776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46477f-5179-4723-959d-e7d16b427ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:12.178292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46477f-5179-4723-959d-e7d16b427ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:12.180647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46477f-5179-4723-959d-e7d16b427ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:12.182019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46477f-5179-4723-959d-e7d16b427ce0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:12.186389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:12.203734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 949/1909] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:12.205378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46477f-5179-4723-959d-e7d16b427ce0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:12.205769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 961/1910] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:12 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:12.206685 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d3e99-d39e-4e74-8b0a-06e6fb704e35 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:12.207958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5a46477f-5179-4723-959d-e7d16b427ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:12.292281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46477f-5179-4723-959d-e7d16b427ce0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:12.335791 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d3e99-d39e-4e74-8b0a-06e6fb704e35 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:12.343884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0afe1-7848-49d7-8a84-9420b66377b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:12.344691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0afe1-7848-49d7-8a84-9420b66377b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:12.344840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0afe1-7848-49d7-8a84-9420b66377b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:12.345086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0afe1-7848-49d7-8a84-9420b66377b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:12.345745 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0afe1-7848-49d7-8a84-9420b66377b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:12.348014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0afe1-7848-49d7-8a84-9420b66377b0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:12.387338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 962/1911] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:12.388498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0afe1-7848-49d7-8a84-9420b66377b0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:12.403958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0afe1-7848-49d7-8a84-9420b66377b0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:12.410589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37b79c-1ebb-4222-aea8-a7774a5c1e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:12.411412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37b79c-1ebb-4222-aea8-a7774a5c1e53 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:12.413424 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37b79c-1ebb-4222-aea8-a7774a5c1e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:12.413756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37b79c-1ebb-4222-aea8-a7774a5c1e53 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:12.415363 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37b79c-1ebb-4222-aea8-a7774a5c1e53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:12.418668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37b79c-1ebb-4222-aea8-a7774a5c1e53 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:12.420334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fa459-78e4-445e-9be2-a638a18e6a04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:12.420989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fa459-78e4-445e-9be2-a638a18e6a04 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:12.421271 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fa459-78e4-445e-9be2-a638a18e6a04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:12.421547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fa459-78e4-445e-9be2-a638a18e6a04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:12.457650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 963/1912] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:12 2022] POST /identity/v3/users => generated 287 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:12.458602 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37b79c-1ebb-4222-aea8-a7774a5c1e53 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:12.491248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37b79c-1ebb-4222-aea8-a7774a5c1e53 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:12.509840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d487ec-8190-4181-8bd5-981e4faff78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:12.510732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d487ec-8190-4181-8bd5-981e4faff78f admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:12.511120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d487ec-8190-4181-8bd5-981e4faff78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:12.513310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d487ec-8190-4181-8bd5-981e4faff78f admin admin] PATH_INFO: `/v3/projects/1fbebdb527ce42fca62d9da743135659/users/f244c85df17e4a47aa6b63d032f19abb/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:12.519982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:12.555425 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d487ec-8190-4181-8bd5-981e4faff78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fbebdb527ce42fca62d9da743135659, user_id=f244c85df17e4a47aa6b63d032f19abb, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:12.558530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d487ec-8190-4181-8bd5-981e4faff78f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:12.588514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 964/1913] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:12 2022] PUT /identity/v3/projects/1fbebdb527ce42fca62d9da743135659/users/f244c85df17e4a47aa6b63d032f19abb/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:12.595331 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d487ec-8190-4181-8bd5-981e4faff78f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:12.600611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fa459-78e4-445e-9be2-a638a18e6a04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:12.606298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fa459-78e4-445e-9be2-a638a18e6a04 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:12.643503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 950/1914] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:12.649579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de10b6-da39-4490-b529-af83d52021d6 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:12.649988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de10b6-da39-4490-b529-af83d52021d6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:12.650401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de10b6-da39-4490-b529-af83d52021d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:12.713896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a2de10b6-da39-4490-b529-af83d52021d6 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:12.718083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a2de10b6-da39-4490-b529-af83d52021d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:12.786280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d487ec-8190-4181-8bd5-981e4faff78f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:12.809556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5f2ca-6010-412e-a873-e98ce2a996e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:12.815071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5f2ca-6010-412e-a873-e98ce2a996e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:12.815367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5f2ca-6010-412e-a873-e98ce2a996e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:12.818629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5f2ca-6010-412e-a873-e98ce2a996e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:12.819730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5f2ca-6010-412e-a873-e98ce2a996e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:12.821491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5f2ca-6010-412e-a873-e98ce2a996e3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:12.850105 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 965/1915] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:12.858290 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5f2ca-6010-412e-a873-e98ce2a996e3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:12.885466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5f2ca-6010-412e-a873-e98ce2a996e3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:12.908685 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02188cec-c904-43d4-8425-d05fdc7426f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:12.911796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02188cec-c904-43d4-8425-d05fdc7426f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:12.912088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02188cec-c904-43d4-8425-d05fdc7426f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:12.912358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02188cec-c904-43d4-8425-d05fdc7426f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:12.916297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02188cec-c904-43d4-8425-d05fdc7426f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:12.921815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02188cec-c904-43d4-8425-d05fdc7426f0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:12.944989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 951/1916] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:12.949930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de10b6-da39-4490-b529-af83d52021d6 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:12.969907 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 966/1917] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:12.975766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02188cec-c904-43d4-8425-d05fdc7426f0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:12.978349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:12.987223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de10b6-da39-4490-b529-af83d52021d6 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:13.007415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02188cec-c904-43d4-8425-d05fdc7426f0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:13.012448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89581711-2a60-43e8-a0c3-f75f989ef630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:13.013150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89581711-2a60-43e8-a0c3-f75f989ef630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:13.013403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89581711-2a60-43e8-a0c3-f75f989ef630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:13.016199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89581711-2a60-43e8-a0c3-f75f989ef630 admin admin] PATH_INFO: `/v3/users/45890d0529dd4f8586c288d5713f4bdc` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:13.017751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89581711-2a60-43e8-a0c3-f75f989ef630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45890d0529dd4f8586c288d5713f4bdc)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:13.019808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89581711-2a60-43e8-a0c3-f75f989ef630 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:13.037653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9658c-98de-4fa9-bccf-20f1c725923f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:13.038388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9658c-98de-4fa9-bccf-20f1c725923f admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:13.038672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9658c-98de-4fa9-bccf-20f1c725923f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:13.038977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9658c-98de-4fa9-bccf-20f1c725923f admin admin] PATH_INFO: `/v3/projects/7b447ea1184f4af39569b7ca5a722f65/users/4ea18dbd69064e90bb2dcef5570cb417/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:13.081261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9658c-98de-4fa9-bccf-20f1c725923f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b447ea1184f4af39569b7ca5a722f65, user_id=4ea18dbd69064e90bb2dcef5570cb417, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:13.084255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9658c-98de-4fa9-bccf-20f1c725923f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:13.110315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 967/1918] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:12 2022] PUT /identity/v3/projects/7b447ea1184f4af39569b7ca5a722f65/users/4ea18dbd69064e90bb2dcef5570cb417/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:13.111332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9658c-98de-4fa9-bccf-20f1c725923f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:13.117823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-97c9658c-98de-4fa9-bccf-20f1c725923f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:13.150795 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89581711-2a60-43e8-a0c3-f75f989ef630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45890d0529dd4f8586c288d5713f4bdc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:13.171568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89581711-2a60-43e8-a0c3-f75f989ef630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45890d0529dd4f8586c288d5713f4bdc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:13.181278 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89581711-2a60-43e8-a0c3-f75f989ef630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45890d0529dd4f8586c288d5713f4bdc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:13.182045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89581711-2a60-43e8-a0c3-f75f989ef630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45890d0529dd4f8586c288d5713f4bdc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:13.192272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89581711-2a60-43e8-a0c3-f75f989ef630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45890d0529dd4f8586c288d5713f4bdc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:13.206830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 952/1919] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:12 2022] DELETE /identity/v3/users/45890d0529dd4f8586c288d5713f4bdc => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:13.209430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3bd61-5640-44c8-86a5-d58e134e5fa3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:13.209905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3bd61-5640-44c8-86a5-d58e134e5fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:13.210312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3bd61-5640-44c8-86a5-d58e134e5fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:13.229179 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9658c-98de-4fa9-bccf-20f1c725923f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:13.248560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-49c3bd61-5640-44c8-86a5-d58e134e5fa3 None None] MFA Rules not processed for user `4ea18dbd69064e90bb2dcef5570cb417`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:13.249791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-49c3bd61-5640-44c8-86a5-d58e134e5fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:13.294753 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546659c-fb2a-4e8e-9f6d-7e86b6c66cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:13.295419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546659c-fb2a-4e8e-9f6d-7e86b6c66cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:13.295696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546659c-fb2a-4e8e-9f6d-7e86b6c66cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:13.295973 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546659c-fb2a-4e8e-9f6d-7e86b6c66cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:13.298375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2546659c-fb2a-4e8e-9f6d-7e86b6c66cf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:13.317976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:13.336731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 953/1920] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:13.343294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83734b3-1725-45a0-bb92-4d615c3cc549 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:13.343294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83734b3-1725-45a0-bb92-4d615c3cc549 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:13.343294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83734b3-1725-45a0-bb92-4d615c3cc549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:13.380517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b83734b3-1725-45a0-bb92-4d615c3cc549 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:13.382116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b83734b3-1725-45a0-bb92-4d615c3cc549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:13.414145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546659c-fb2a-4e8e-9f6d-7e86b6c66cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:13.416124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546659c-fb2a-4e8e-9f6d-7e86b6c66cf6 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:13.433819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 968/1921] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:13.434938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546659c-fb2a-4e8e-9f6d-7e86b6c66cf6 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:13.436518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2546659c-fb2a-4e8e-9f6d-7e86b6c66cf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:13.523581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 954/1922] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:13.544018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546659c-fb2a-4e8e-9f6d-7e86b6c66cf6 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:13.549667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144004fe-f11d-49ef-8b74-b310cf37f4ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:13.550810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144004fe-f11d-49ef-8b74-b310cf37f4ff service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:13.551267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144004fe-f11d-49ef-8b74-b310cf37f4ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:13.551688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144004fe-f11d-49ef-8b74-b310cf37f4ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:13.559976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83734b3-1725-45a0-bb92-4d615c3cc549 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:13.562710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b83734b3-1725-45a0-bb92-4d615c3cc549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:13.566304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144004fe-f11d-49ef-8b74-b310cf37f4ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:13.568119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144004fe-f11d-49ef-8b74-b310cf37f4ff service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:13.596491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 969/1923] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:13.623149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144004fe-f11d-49ef-8b74-b310cf37f4ff service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:13.625212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-144004fe-f11d-49ef-8b74-b310cf37f4ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:13.625970 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83734b3-1725-45a0-bb92-4d615c3cc549 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:13.632675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c464bfa-aef4-4cee-8157-e0fa37aee2f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:13.633918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464bfa-aef4-4cee-8157-e0fa37aee2f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:13.634570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464bfa-aef4-4cee-8157-e0fa37aee2f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:13.635183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464bfa-aef4-4cee-8157-e0fa37aee2f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:13.647225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:13.662185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144004fe-f11d-49ef-8b74-b310cf37f4ff service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:13.663755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c464bfa-aef4-4cee-8157-e0fa37aee2f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:13.666560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c464bfa-aef4-4cee-8157-e0fa37aee2f5 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:13.671074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:13.672472 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:13.673076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:13.673681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] PATH_INFO: `/v3/users/4ea18dbd69064e90bb2dcef5570cb417` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:13.676274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ea18dbd69064e90bb2dcef5570cb417)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:13.680677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:13.705240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 955/1924] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:13.815732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ea18dbd69064e90bb2dcef5570cb417'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:13.841652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ea18dbd69064e90bb2dcef5570cb417'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:13.850568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ea18dbd69064e90bb2dcef5570cb417'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:13.861401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ea18dbd69064e90bb2dcef5570cb417'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:13.862346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ea18dbd69064e90bb2dcef5570cb417'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:13.873508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 970/1925] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:13 2022] DELETE /identity/v3/users/4ea18dbd69064e90bb2dcef5570cb417 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:13.878363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302c61b-f4b8-46da-a1de-38016095eb6b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:13.878676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302c61b-f4b8-46da-a1de-38016095eb6b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:13.878939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302c61b-f4b8-46da-a1de-38016095eb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:13.904533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3302c61b-f4b8-46da-a1de-38016095eb6b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:13.905894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3302c61b-f4b8-46da-a1de-38016095eb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:14.022798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 956/1926] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:14.078528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:14.080553 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:14.146476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:14.201347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5acc75-8f94-4f29-af89-434dff5e60c8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:14.205657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef9039-55d8-47cf-b6ba-dbc377fda66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:14.206292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef9039-55d8-47cf-b6ba-dbc377fda66e service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:14.206623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef9039-55d8-47cf-b6ba-dbc377fda66e service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:14.207090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef9039-55d8-47cf-b6ba-dbc377fda66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:14.219669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef9039-55d8-47cf-b6ba-dbc377fda66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:14.221439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef9039-55d8-47cf-b6ba-dbc377fda66e service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:14.235152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 971/1927] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:14.762900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b4d66-a5f2-4f5d-8cb9-b9ae5b69df30 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:14.763716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b4d66-a5f2-4f5d-8cb9-b9ae5b69df30 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:14.764210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b4d66-a5f2-4f5d-8cb9-b9ae5b69df30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:14.804000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4f8b4d66-a5f2-4f5d-8cb9-b9ae5b69df30 None None] MFA Rules not processed for user `f244c85df17e4a47aa6b63d032f19abb`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:14.805831 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4f8b4d66-a5f2-4f5d-8cb9-b9ae5b69df30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:14.891232 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 957/1928] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:14.896292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef9039-55d8-47cf-b6ba-dbc377fda66e service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:14.897852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-04ef9039-55d8-47cf-b6ba-dbc377fda66e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:14.911501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef9039-55d8-47cf-b6ba-dbc377fda66e service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:14.918532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d44f49-523f-4f32-9c82-f0552787bf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:14.919846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44f49-523f-4f32-9c82-f0552787bf25 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:14.920911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44f49-523f-4f32-9c82-f0552787bf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:14.921501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44f49-523f-4f32-9c82-f0552787bf25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:14.922206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d44f49-523f-4f32-9c82-f0552787bf25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:14.924990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d44f49-523f-4f32-9c82-f0552787bf25 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:14.963032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 972/1929] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:14 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:14.966803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b4d66-a5f2-4f5d-8cb9-b9ae5b69df30 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:14.982599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b4d66-a5f2-4f5d-8cb9-b9ae5b69df30 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:15.044900 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d44f49-523f-4f32-9c82-f0552787bf25 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:15.050662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-00d44f49-523f-4f32-9c82-f0552787bf25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:15.148858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912497be-b355-4f11-be24-2be195edb5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:15.149682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912497be-b355-4f11-be24-2be195edb5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:15.150086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912497be-b355-4f11-be24-2be195edb5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:15.150465 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912497be-b355-4f11-be24-2be195edb5d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:15.151315 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912497be-b355-4f11-be24-2be195edb5d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:15.154910 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912497be-b355-4f11-be24-2be195edb5d5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:15.204870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d44f49-523f-4f32-9c82-f0552787bf25 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:15.208954 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 958/1930] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:14 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:15.210519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912497be-b355-4f11-be24-2be195edb5d5 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:15.220221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:15.220712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:15.221202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:15.221746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] PATH_INFO: `/v3/projects/39a091e948b64d4ea93bf2075f7c65ba` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:15.224243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39a091e948b64d4ea93bf2075f7c65ba)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:15.228943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:15.233396 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912497be-b355-4f11-be24-2be195edb5d5 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:15.253469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e4b76-883d-4bda-8549-17a3ff2e1ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:15.254884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4b76-883d-4bda-8549-17a3ff2e1ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:15.255505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4b76-883d-4bda-8549-17a3ff2e1ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:15.257481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4b76-883d-4bda-8549-17a3ff2e1ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:15.258752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e4b76-883d-4bda-8549-17a3ff2e1ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:15.267581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e4b76-883d-4bda-8549-17a3ff2e1ac6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:15.302203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 959/1931] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:15.304788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e4b76-883d-4bda-8549-17a3ff2e1ac6 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:15.317931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:15.336281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e4b76-883d-4bda-8549-17a3ff2e1ac6 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:15.358790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39a091e948b64d4ea93bf2075f7c65ba', 'request_id': 'req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.373514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39a091e948b64d4ea93bf2075f7c65ba', 'request_id': 'req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.397501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39a091e948b64d4ea93bf2075f7c65ba', 'request_id': 'req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.429334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] The token cache is being invalidate because project 39a091e948b64d4ea93bf2075f7c65ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:15.429821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.431015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.440593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 973/1932] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:15 2022] DELETE /identity/v3/projects/39a091e948b64d4ea93bf2075f7c65ba => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:15.443036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:15.445706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:15.476535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a5ceb-9d59-4e3b-9837-ff6f521afde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:15.477861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a5ceb-9d59-4e3b-9837-ff6f521afde6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:15.478403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a5ceb-9d59-4e3b-9837-ff6f521afde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:15.479041 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a5ceb-9d59-4e3b-9837-ff6f521afde6 admin admin] PATH_INFO: `/v3/projects/568cf0f015a442fcbe5cd769fa775665/users/b07f8d0748984ef5ad47422b3e73e467/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:15.511860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a5ceb-9d59-4e3b-9837-ff6f521afde6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568cf0f015a442fcbe5cd769fa775665, user_id=b07f8d0748984ef5ad47422b3e73e467, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:15.519612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a5ceb-9d59-4e3b-9837-ff6f521afde6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:15.559089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 960/1933] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:15 2022] PUT /identity/v3/projects/568cf0f015a442fcbe5cd769fa775665/users/b07f8d0748984ef5ad47422b3e73e467/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:15.559089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a5ceb-9d59-4e3b-9837-ff6f521afde6 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:15.562186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-367a5ceb-9d59-4e3b-9837-ff6f521afde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:15.565177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:15.640464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e2d7b-5b06-4c36-bbc3-610e10dbddd2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:15.675576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a5ceb-9d59-4e3b-9837-ff6f521afde6 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:15.683048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:15.684061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:15.684587 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:15.685011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] PATH_INFO: `/v3/projects/7b447ea1184f4af39569b7ca5a722f65` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:15.686938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b447ea1184f4af39569b7ca5a722f65)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:15.690007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:15.727572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3aac0-9ab2-4bef-80dc-436e8816be0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:15.728564 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3aac0-9ab2-4bef-80dc-436e8816be0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:15.728847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3aac0-9ab2-4bef-80dc-436e8816be0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:15.729272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3aac0-9ab2-4bef-80dc-436e8816be0f admin admin] PATH_INFO: `/v3/users/be45d39363a04b3e8da509a307ce6927` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:15.732290 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3aac0-9ab2-4bef-80dc-436e8816be0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=be45d39363a04b3e8da509a307ce6927)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:15.735239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3aac0-9ab2-4bef-80dc-436e8816be0f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:15.788890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b447ea1184f4af39569b7ca5a722f65', 'request_id': 'req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.809358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b447ea1184f4af39569b7ca5a722f65', 'request_id': 'req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.816106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b447ea1184f4af39569b7ca5a722f65', 'request_id': 'req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.826695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67e3aac0-9ab2-4bef-80dc-436e8816be0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be45d39363a04b3e8da509a307ce6927'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.832253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] The token cache is being invalidate because project 7b447ea1184f4af39569b7ca5a722f65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:15.832672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.833344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.835793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 961/1934] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:15 2022] DELETE /identity/v3/projects/7b447ea1184f4af39569b7ca5a722f65 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:15.836251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:15.840733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:15.849692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67e3aac0-9ab2-4bef-80dc-436e8816be0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be45d39363a04b3e8da509a307ce6927'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.860358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67e3aac0-9ab2-4bef-80dc-436e8816be0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be45d39363a04b3e8da509a307ce6927'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.872149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:15.875919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67e3aac0-9ab2-4bef-80dc-436e8816be0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be45d39363a04b3e8da509a307ce6927'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.876890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67e3aac0-9ab2-4bef-80dc-436e8816be0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be45d39363a04b3e8da509a307ce6927'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:15.884808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 974/1935] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:15 2022] DELETE /identity/v3/users/be45d39363a04b3e8da509a307ce6927 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:15.887004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aba33-ce35-4ec0-b514-ecc3ca0b7e65 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:15.887469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aba33-ce35-4ec0-b514-ecc3ca0b7e65 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:15.888063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aba33-ce35-4ec0-b514-ecc3ca0b7e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:15.958332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-771aba33-ce35-4ec0-b514-ecc3ca0b7e65 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:15.967704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-771aba33-ce35-4ec0-b514-ecc3ca0b7e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:15.987702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef0e36-4ad4-42b7-92ca-a358fd8e82c9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:16.125919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab886f9-9659-4abf-8a63-c05e1c4195eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:16.127858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab886f9-9659-4abf-8a63-c05e1c4195eb admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:16.128366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab886f9-9659-4abf-8a63-c05e1c4195eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:16.128866 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab886f9-9659-4abf-8a63-c05e1c4195eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:16.130566 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab886f9-9659-4abf-8a63-c05e1c4195eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:16.134151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab886f9-9659-4abf-8a63-c05e1c4195eb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:16.156959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 962/1936] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:16.162382 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 975/1937] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:16.163492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab886f9-9659-4abf-8a63-c05e1c4195eb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:16.175724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3889934-a74c-4413-a4ac-dc8dec7901ac None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:16.176085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3889934-a74c-4413-a4ac-dc8dec7901ac None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:16.176429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3889934-a74c-4413-a4ac-dc8dec7901ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:16.177733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7ab886f9-9659-4abf-8a63-c05e1c4195eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:16.191381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:16.193140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab886f9-9659-4abf-8a63-c05e1c4195eb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:16.198684 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39029664-c02c-41c7-9a43-75418d083d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:16.199845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39029664-c02c-41c7-9a43-75418d083d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:16.200059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39029664-c02c-41c7-9a43-75418d083d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:16.200341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39029664-c02c-41c7-9a43-75418d083d4a admin admin] PATH_INFO: `/v3/projects/568cf0f015a442fcbe5cd769fa775665/users/b07f8d0748984ef5ad47422b3e73e467/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:16.202632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39029664-c02c-41c7-9a43-75418d083d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=568cf0f015a442fcbe5cd769fa775665, user_id=b07f8d0748984ef5ad47422b3e73e467, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:16.206601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39029664-c02c-41c7-9a43-75418d083d4a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:16.220988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a3889934-a74c-4413-a4ac-dc8dec7901ac None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:16.222542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a3889934-a74c-4413-a4ac-dc8dec7901ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:16.239762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 963/1938] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:16 2022] PUT /identity/v3/projects/568cf0f015a442fcbe5cd769fa775665/users/b07f8d0748984ef5ad47422b3e73e467/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:16.391974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 976/1939] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:16.392848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39029664-c02c-41c7-9a43-75418d083d4a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:16.439086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39029664-c02c-41c7-9a43-75418d083d4a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:16.442466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3889934-a74c-4413-a4ac-dc8dec7901ac None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:16.446010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a3889934-a74c-4413-a4ac-dc8dec7901ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:16.469759 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aafc9f-2187-478a-bca3-b9b8d01cb2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:16.474234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aafc9f-2187-478a-bca3-b9b8d01cb2fa admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:16.475358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aafc9f-2187-478a-bca3-b9b8d01cb2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:16.476651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aafc9f-2187-478a-bca3-b9b8d01cb2fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:16.481749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3889934-a74c-4413-a4ac-dc8dec7901ac None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:16.482543 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aafc9f-2187-478a-bca3-b9b8d01cb2fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:16.483976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aafc9f-2187-478a-bca3-b9b8d01cb2fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:16.491261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aafc9f-2187-478a-bca3-b9b8d01cb2fa admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:16.504610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:16.505898 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:16.508129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:16.511583 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] PATH_INFO: `/v3/projects/15a416986eec4ed58986ccd74ef31fff` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:16.517008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a416986eec4ed58986ccd74ef31fff)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:16.521376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:16.531007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 964/1940] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 17:08:16.540411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aafc9f-2187-478a-bca3-b9b8d01cb2fa admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:16.568280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aafc9f-2187-478a-bca3-b9b8d01cb2fa admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:16.586244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e9726-fee0-4f3f-8ca3-141fa13c1c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:16.589666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e9726-fee0-4f3f-8ca3-141fa13c1c97 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:16.590052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e9726-fee0-4f3f-8ca3-141fa13c1c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:16.591059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e9726-fee0-4f3f-8ca3-141fa13c1c97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:16.594072 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e9726-fee0-4f3f-8ca3-141fa13c1c97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:16.599284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e9726-fee0-4f3f-8ca3-141fa13c1c97 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:16.653108 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a416986eec4ed58986ccd74ef31fff', 'request_id': 'req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:16.662102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a416986eec4ed58986ccd74ef31fff', 'request_id': 'req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:16.674728 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 965/1941] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:16 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:16.681624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e9726-fee0-4f3f-8ca3-141fa13c1c97 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:16.683699 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a416986eec4ed58986ccd74ef31fff', 'request_id': 'req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:16.713023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] The token cache is being invalidate because project 15a416986eec4ed58986ccd74ef31fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:16.713430 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:16.715172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d64ba1b6-6bd2-461c-81e8-e200a4fbfae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:16.722152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 977/1942] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:16 2022] DELETE /identity/v3/projects/15a416986eec4ed58986ccd74ef31fff => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:16.726009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6e142-4397-476a-b8e0-531a9e1e5552 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:16.726411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6e142-4397-476a-b8e0-531a9e1e5552 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:16.727436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6e142-4397-476a-b8e0-531a9e1e5552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:16.786920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-94d6e142-4397-476a-b8e0-531a9e1e5552 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:16.795243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-94d6e142-4397-476a-b8e0-531a9e1e5552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:16.826797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e9726-fee0-4f3f-8ca3-141fa13c1c97 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:16.827320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 978/1943] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:16.835404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758ee99-90f6-4167-8900-6f9a40f34f6c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:16.835786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758ee99-90f6-4167-8900-6f9a40f34f6c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:16.836859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758ee99-90f6-4167-8900-6f9a40f34f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:16.842252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac881fd-71e3-4a34-aa33-2128fb07843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:16.848000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac881fd-71e3-4a34-aa33-2128fb07843e admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:16.848297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac881fd-71e3-4a34-aa33-2128fb07843e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:16.850771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac881fd-71e3-4a34-aa33-2128fb07843e admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:16.851510 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac881fd-71e3-4a34-aa33-2128fb07843e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:16.856051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac881fd-71e3-4a34-aa33-2128fb07843e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:16.864873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5758ee99-90f6-4167-8900-6f9a40f34f6c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:16.867043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5758ee99-90f6-4167-8900-6f9a40f34f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:16.880058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 979/1944] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:16.884202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758ee99-90f6-4167-8900-6f9a40f34f6c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:16.897212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:16.910001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 966/1945] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:16 2022] POST /identity/v3/users => generated 287 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:16.911669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758ee99-90f6-4167-8900-6f9a40f34f6c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:16.913712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac881fd-71e3-4a34-aa33-2128fb07843e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:16.917762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fac881fd-71e3-4a34-aa33-2128fb07843e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:16.925020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da95756-22fa-4562-8478-d84f681b3b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:16.927116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da95756-22fa-4562-8478-d84f681b3b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:16.928504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da95756-22fa-4562-8478-d84f681b3b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:16.928885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da95756-22fa-4562-8478-d84f681b3b34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:16.930550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da95756-22fa-4562-8478-d84f681b3b34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:16.933363 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da95756-22fa-4562-8478-d84f681b3b34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:16.938896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da95756-22fa-4562-8478-d84f681b3b34 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:16.964652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac881fd-71e3-4a34-aa33-2128fb07843e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:16.979455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb706b-0632-4d04-957e-f7d6150fa6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:16.980859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb706b-0632-4d04-957e-f7d6150fa6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:16.981665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb706b-0632-4d04-957e-f7d6150fa6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:16.981665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb706b-0632-4d04-957e-f7d6150fa6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:16.985253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb706b-0632-4d04-957e-f7d6150fa6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:16.990643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 980/1946] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:08:16.990839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb706b-0632-4d04-957e-f7d6150fa6e9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:16.992767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da95756-22fa-4562-8478-d84f681b3b34 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:17.018410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da95756-22fa-4562-8478-d84f681b3b34 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:17.025088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 967/1947] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:17.025388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f766894-2b68-42cf-8338-99334825368b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:17.026132 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb706b-0632-4d04-957e-f7d6150fa6e9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:17.026296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f766894-2b68-42cf-8338-99334825368b admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:17.026645 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f766894-2b68-42cf-8338-99334825368b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:17.027086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f766894-2b68-42cf-8338-99334825368b admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:17.028033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f766894-2b68-42cf-8338-99334825368b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:17.031041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f766894-2b68-42cf-8338-99334825368b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:17.081225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 981/1948] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:16 2022] POST /identity/v3/projects => generated 409 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:17.090242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f766894-2b68-42cf-8338-99334825368b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:17.092774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb706b-0632-4d04-957e-f7d6150fa6e9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:17.196447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01f5f4-e584-4c3a-a3a7-0d76993a8e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:17.198374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01f5f4-e584-4c3a-a3a7-0d76993a8e55 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:17.198374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01f5f4-e584-4c3a-a3a7-0d76993a8e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:17.199489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01f5f4-e584-4c3a-a3a7-0d76993a8e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:17.201639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01f5f4-e584-4c3a-a3a7-0d76993a8e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:17.204872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01f5f4-e584-4c3a-a3a7-0d76993a8e55 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:17.230381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 968/1949] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:08:17.235885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01f5f4-e584-4c3a-a3a7-0d76993a8e55 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:17.245013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:17.258389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01f5f4-e584-4c3a-a3a7-0d76993a8e55 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:17.266283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd97b8-7145-40d1-905d-8181706718de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:17.268216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd97b8-7145-40d1-905d-8181706718de admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:17.268735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd97b8-7145-40d1-905d-8181706718de admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:17.269281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd97b8-7145-40d1-905d-8181706718de admin admin] PATH_INFO: `/v3/projects/469c94da6b974013a0a546d254e7f160/users/09641f97860b47d7bea8fa1396c44288/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:17.309658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd97b8-7145-40d1-905d-8181706718de admin admin] RBAC: Authorizing `identity:create_grant(project_id=469c94da6b974013a0a546d254e7f160, user_id=09641f97860b47d7bea8fa1396c44288, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:17.313268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd97b8-7145-40d1-905d-8181706718de admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:17.332278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f766894-2b68-42cf-8338-99334825368b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:17.351140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 969/1950] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:17 2022] PUT /identity/v3/projects/469c94da6b974013a0a546d254e7f160/users/09641f97860b47d7bea8fa1396c44288/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:17.369037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61a7b-0bbf-4bd6-911b-ff424302809d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:17.369687 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61a7b-0bbf-4bd6-911b-ff424302809d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:17.370041 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61a7b-0bbf-4bd6-911b-ff424302809d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:17.465852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4fe61a7b-0bbf-4bd6-911b-ff424302809d None None] MFA Rules not processed for user `09641f97860b47d7bea8fa1396c44288`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:17.470696 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4fe61a7b-0bbf-4bd6-911b-ff424302809d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:17.554036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be1ddd-8410-4674-96da-5c9a074c8066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:17.555306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1ddd-8410-4674-96da-5c9a074c8066 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:17.555306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1ddd-8410-4674-96da-5c9a074c8066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:17.555894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1ddd-8410-4674-96da-5c9a074c8066 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:17.556536 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be1ddd-8410-4674-96da-5c9a074c8066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:17.562666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be1ddd-8410-4674-96da-5c9a074c8066 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:17.618479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 982/1951] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:17 2022] POST /identity/v3/users => generated 305 bytes in 529 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:17.623327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be1ddd-8410-4674-96da-5c9a074c8066 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:17.644683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be1ddd-8410-4674-96da-5c9a074c8066 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:17.673061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c3800-4b4b-4da3-b81e-68ce76eb27c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:17.673288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 970/1952] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:17.680741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c3800-4b4b-4da3-b81e-68ce76eb27c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:17.681860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c3800-4b4b-4da3-b81e-68ce76eb27c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:17.685712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c3800-4b4b-4da3-b81e-68ce76eb27c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:17.687315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c3800-4b4b-4da3-b81e-68ce76eb27c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:17.704407 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe61a7b-0bbf-4bd6-911b-ff424302809d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:17.706672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4fe61a7b-0bbf-4bd6-911b-ff424302809d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:17.719981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c3800-4b4b-4da3-b81e-68ce76eb27c2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:17.762045 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 983/1953] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:08:17.767293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c3800-4b4b-4da3-b81e-68ce76eb27c2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:17.801472 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c3800-4b4b-4da3-b81e-68ce76eb27c2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:17.818185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b7013-8e13-4331-b5ce-86a3f52ea45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:17.819204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b7013-8e13-4331-b5ce-86a3f52ea45d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:17.819726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b7013-8e13-4331-b5ce-86a3f52ea45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:17.820194 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b7013-8e13-4331-b5ce-86a3f52ea45d admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:17.821295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b7013-8e13-4331-b5ce-86a3f52ea45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:17.823405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b7013-8e13-4331-b5ce-86a3f52ea45d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:17.847002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 984/1954] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:17.850779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b7013-8e13-4331-b5ce-86a3f52ea45d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:17.870883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe61a7b-0bbf-4bd6-911b-ff424302809d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:17.877089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9bbc-9fe3-4fe3-8053-1f2ded7b42eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:17.878195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9bbc-9fe3-4fe3-8053-1f2ded7b42eb service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:17.878876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9bbc-9fe3-4fe3-8053-1f2ded7b42eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:17.879649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9bbc-9fe3-4fe3-8053-1f2ded7b42eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:17.880497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b7013-8e13-4331-b5ce-86a3f52ea45d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:17.895379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:17.898085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb484b-72ea-4cf0-9328-cad7050aa7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:17.898783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb484b-72ea-4cf0-9328-cad7050aa7d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:17.899137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb484b-72ea-4cf0-9328-cad7050aa7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:17.899430 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb484b-72ea-4cf0-9328-cad7050aa7d9 admin admin] PATH_INFO: `/v3/projects/66cfd185e94645078db1df52c047cfae/users/f11e023666434a619b4fe94724ace8f2/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:17.920034 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca9bbc-9fe3-4fe3-8053-1f2ded7b42eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:17.928842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca9bbc-9fe3-4fe3-8053-1f2ded7b42eb service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:17.972112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb484b-72ea-4cf0-9328-cad7050aa7d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66cfd185e94645078db1df52c047cfae, user_id=f11e023666434a619b4fe94724ace8f2, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:17.975087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 971/1955] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:17.986610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb484b-72ea-4cf0-9328-cad7050aa7d9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:18.027716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 985/1956] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:17 2022] PUT /identity/v3/projects/66cfd185e94645078db1df52c047cfae/users/f11e023666434a619b4fe94724ace8f2/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:18.033635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23297083-4bef-4e0e-a9ed-5a1f7a1b0f37 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:18.033931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23297083-4bef-4e0e-a9ed-5a1f7a1b0f37 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:18.034253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23297083-4bef-4e0e-a9ed-5a1f7a1b0f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:18.113730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-23297083-4bef-4e0e-a9ed-5a1f7a1b0f37 None None] MFA Rules not processed for user `f11e023666434a619b4fe94724ace8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:18.115023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-23297083-4bef-4e0e-a9ed-5a1f7a1b0f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:18.294749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 972/1957] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:18 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:18.301578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb484b-72ea-4cf0-9328-cad7050aa7d9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:18.524977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925b5a-e1fb-4142-bf6a-615729a90332 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:18.525288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925b5a-e1fb-4142-bf6a-615729a90332 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:18.525632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925b5a-e1fb-4142-bf6a-615729a90332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:18.599895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb484b-72ea-4cf0-9328-cad7050aa7d9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:18.614273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a59448-d77e-4dd8-8513-85398a6bfafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:18.614491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-37925b5a-e1fb-4142-bf6a-615729a90332 None None] MFA Rules not processed for user `b07f8d0748984ef5ad47422b3e73e467`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:18.614576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a59448-d77e-4dd8-8513-85398a6bfafc admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:18.614793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a59448-d77e-4dd8-8513-85398a6bfafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:18.615118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a59448-d77e-4dd8-8513-85398a6bfafc admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:18.615983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-37925b5a-e1fb-4142-bf6a-615729a90332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:18.621686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a59448-d77e-4dd8-8513-85398a6bfafc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:18.627333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a59448-d77e-4dd8-8513-85398a6bfafc admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:18.699168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 986/1958] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:18 2022] POST /identity/v3/projects => generated 407 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:18.702007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a59448-d77e-4dd8-8513-85398a6bfafc admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:18.737877 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:18.847508 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 973/1959] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:18.858672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37925b5a-e1fb-4142-bf6a-615729a90332 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:18.979173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37925b5a-e1fb-4142-bf6a-615729a90332 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:18.981117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a59448-d77e-4dd8-8513-85398a6bfafc admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:18.991725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc06791-aa94-43fd-9aec-ae48063c0cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:18.992363 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc06791-aa94-43fd-9aec-ae48063c0cba admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:18.992717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc06791-aa94-43fd-9aec-ae48063c0cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:18.993289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc06791-aa94-43fd-9aec-ae48063c0cba admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:18.993945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc06791-aa94-43fd-9aec-ae48063c0cba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:18.999566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc06791-aa94-43fd-9aec-ae48063c0cba admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:19.037047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:19.068719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 987/1960] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:18 2022] POST /identity/v3/users => generated 310 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:19.072683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc06791-aa94-43fd-9aec-ae48063c0cba admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:19.092773 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc06791-aa94-43fd-9aec-ae48063c0cba admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:19.098856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5436d6-d6ff-42cb-bab5-44ae3e6e8a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:19.099662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5436d6-d6ff-42cb-bab5-44ae3e6e8a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:19.099944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5436d6-d6ff-42cb-bab5-44ae3e6e8a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:19.100208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5436d6-d6ff-42cb-bab5-44ae3e6e8a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:19.100880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5436d6-d6ff-42cb-bab5-44ae3e6e8a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:19.102690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5436d6-d6ff-42cb-bab5-44ae3e6e8a07 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:19.106508 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636cb08-8c07-448d-a1e6-a602f5f8499e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:19.107251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636cb08-8c07-448d-a1e6-a602f5f8499e service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:19.107773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636cb08-8c07-448d-a1e6-a602f5f8499e service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:19.108060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636cb08-8c07-448d-a1e6-a602f5f8499e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:19.110166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b636cb08-8c07-448d-a1e6-a602f5f8499e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:19.122384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 988/1961] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:19.127341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5436d6-d6ff-42cb-bab5-44ae3e6e8a07 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:19.137944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5436d6-d6ff-42cb-bab5-44ae3e6e8a07 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:19.143063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dd92e-584a-486e-924f-5d8f94b5fe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:19.143789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dd92e-584a-486e-924f-5d8f94b5fe1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:19.144047 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dd92e-584a-486e-924f-5d8f94b5fe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:19.144320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dd92e-584a-486e-924f-5d8f94b5fe1d admin admin] PATH_INFO: `/v3/projects/335600a391044a94a709fc70cce59278/users/e73c44a18c774f76bb8768e7f1e336db/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:19.163874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574dd92e-584a-486e-924f-5d8f94b5fe1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=335600a391044a94a709fc70cce59278, user_id=e73c44a18c774f76bb8768e7f1e336db, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:19.166615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574dd92e-584a-486e-924f-5d8f94b5fe1d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:19.185520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 989/1962] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:19 2022] PUT /identity/v3/projects/335600a391044a94a709fc70cce59278/users/e73c44a18c774f76bb8768e7f1e336db/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:19.189605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dd92e-584a-486e-924f-5d8f94b5fe1d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:19.204342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636cb08-8c07-448d-a1e6-a602f5f8499e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:19.206510 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636cb08-8c07-448d-a1e6-a602f5f8499e service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:19.226244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 974/1963] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:08:19.306612 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dd92e-584a-486e-924f-5d8f94b5fe1d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:19.320719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7333363-e469-4004-a398-25376f6b17a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:19.321361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7333363-e469-4004-a398-25376f6b17a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:19.321639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7333363-e469-4004-a398-25376f6b17a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:19.321911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7333363-e469-4004-a398-25376f6b17a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:19.322573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7333363-e469-4004-a398-25376f6b17a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:19.324287 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7333363-e469-4004-a398-25376f6b17a4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:19.337401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 990/1964] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:08:19.342117 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636cb08-8c07-448d-a1e6-a602f5f8499e service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:19.357137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636cb08-8c07-448d-a1e6-a602f5f8499e service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:19.371891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd5fdd-3afc-4fc0-8626-de11cea4a205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:19.372589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd5fdd-3afc-4fc0-8626-de11cea4a205 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:19.372865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd5fdd-3afc-4fc0-8626-de11cea4a205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:19.375927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd5fdd-3afc-4fc0-8626-de11cea4a205 admin admin] PATH_INFO: `/v3/projects/335600a391044a94a709fc70cce59278/users/e73c44a18c774f76bb8768e7f1e336db/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:19.381614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd5fdd-3afc-4fc0-8626-de11cea4a205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=335600a391044a94a709fc70cce59278, user_id=e73c44a18c774f76bb8768e7f1e336db, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:19.387304 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd5fdd-3afc-4fc0-8626-de11cea4a205 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:19.419855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 975/1965] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:19 2022] PUT /identity/v3/projects/335600a391044a94a709fc70cce59278/users/e73c44a18c774f76bb8768e7f1e336db/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:19.431738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30f1bf-8f48-4859-a127-43dc816ab3fd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:19.432741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30f1bf-8f48-4859-a127-43dc816ab3fd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:19.434015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30f1bf-8f48-4859-a127-43dc816ab3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:19.504387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-af30f1bf-8f48-4859-a127-43dc816ab3fd None None] MFA Rules not processed for user `e73c44a18c774f76bb8768e7f1e336db`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:19.508218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-af30f1bf-8f48-4859-a127-43dc816ab3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:19.638271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:19.682528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 991/1966] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:19 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:19.691404 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd5fdd-3afc-4fc0-8626-de11cea4a205 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:19.695277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9efd5fdd-3afc-4fc0-8626-de11cea4a205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:19.805053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd5fdd-3afc-4fc0-8626-de11cea4a205 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:19.809990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10212398-e29c-4ddd-ac3f-98aee77a02ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:19.810620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10212398-e29c-4ddd-ac3f-98aee77a02ea service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:19.811571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10212398-e29c-4ddd-ac3f-98aee77a02ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:19.811571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10212398-e29c-4ddd-ac3f-98aee77a02ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:19.822461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10212398-e29c-4ddd-ac3f-98aee77a02ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:19.824620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10212398-e29c-4ddd-ac3f-98aee77a02ea service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:19.842571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 976/1967] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:19 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:20.043906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30f1bf-8f48-4859-a127-43dc816ab3fd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:20.061918 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30f1bf-8f48-4859-a127-43dc816ab3fd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:20.087882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361120ff-6fae-40f5-961c-efd792aa9e2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:20.089088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361120ff-6fae-40f5-961c-efd792aa9e2e service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:20.089519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361120ff-6fae-40f5-961c-efd792aa9e2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:20.090047 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361120ff-6fae-40f5-961c-efd792aa9e2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:20.183984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361120ff-6fae-40f5-961c-efd792aa9e2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:20.185785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361120ff-6fae-40f5-961c-efd792aa9e2e service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:20.197603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 992/1968] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:20 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:20.857811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10212398-e29c-4ddd-ac3f-98aee77a02ea service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:20.870140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10212398-e29c-4ddd-ac3f-98aee77a02ea service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:20.879740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b03b4-8230-4b19-9e7f-5c4f35e1b2b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:20.880374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b03b4-8230-4b19-9e7f-5c4f35e1b2b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:20.880973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b03b4-8230-4b19-9e7f-5c4f35e1b2b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:20.880973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b03b4-8230-4b19-9e7f-5c4f35e1b2b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:20.976638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b03b4-8230-4b19-9e7f-5c4f35e1b2b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:20.977947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b03b4-8230-4b19-9e7f-5c4f35e1b2b3 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:20.995215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 977/1969] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:30.191792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=33 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:30.984785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:32.967415 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361120ff-6fae-40f5-961c-efd792aa9e2e service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:33.072904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361120ff-6fae-40f5-961c-efd792aa9e2e service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:33.081213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:33.082220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:33.082681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:33.083186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] PATH_INFO: `/v3/users/09641f97860b47d7bea8fa1396c44288` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:33.085209 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09641f97860b47d7bea8fa1396c44288)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:33.089439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:33.197623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09641f97860b47d7bea8fa1396c44288'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:33.239452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09641f97860b47d7bea8fa1396c44288'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:33.253667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09641f97860b47d7bea8fa1396c44288'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:33.269638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09641f97860b47d7bea8fa1396c44288'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:33.270747 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09641f97860b47d7bea8fa1396c44288'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:33.284422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 993/1970] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:32 2022] DELETE /identity/v3/users/09641f97860b47d7bea8fa1396c44288 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:33.290551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23b4ed-2be4-4076-8d7e-126e9d81a479 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:33.291377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23b4ed-2be4-4076-8d7e-126e9d81a479 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:33.291377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23b4ed-2be4-4076-8d7e-126e9d81a479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:33.336675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3c23b4ed-2be4-4076-8d7e-126e9d81a479 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:33.338551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3c23b4ed-2be4-4076-8d7e-126e9d81a479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:33.435431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:33.435895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:33.507752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 978/1971] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:33.528610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23b4ed-2be4-4076-8d7e-126e9d81a479 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:33.532209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3c23b4ed-2be4-4076-8d7e-126e9d81a479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:33.563062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d900a73-9123-4ef2-b8c6-ee5dc9007ba7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:33.575636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048902c-9365-485d-83a2-124c50c57948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:33.576360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048902c-9365-485d-83a2-124c50c57948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:33.576648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048902c-9365-485d-83a2-124c50c57948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:33.579109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048902c-9365-485d-83a2-124c50c57948 admin admin] PATH_INFO: `/v3/users/f11e023666434a619b4fe94724ace8f2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:33.581949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048902c-9365-485d-83a2-124c50c57948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f11e023666434a619b4fe94724ace8f2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:33.590310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048902c-9365-485d-83a2-124c50c57948 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:33.628880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:33.719198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6048902c-9365-485d-83a2-124c50c57948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f11e023666434a619b4fe94724ace8f2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:33.759292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6048902c-9365-485d-83a2-124c50c57948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f11e023666434a619b4fe94724ace8f2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:33.773585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23b4ed-2be4-4076-8d7e-126e9d81a479 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:33.787511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87f13d-54e3-47b2-89ec-c6d0918582b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:33.788571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87f13d-54e3-47b2-89ec-c6d0918582b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:33.788899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87f13d-54e3-47b2-89ec-c6d0918582b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:33.789181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87f13d-54e3-47b2-89ec-c6d0918582b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:33.799834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6048902c-9365-485d-83a2-124c50c57948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f11e023666434a619b4fe94724ace8f2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:33.822551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87f13d-54e3-47b2-89ec-c6d0918582b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:33.826594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6048902c-9365-485d-83a2-124c50c57948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f11e023666434a619b4fe94724ace8f2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:33.827149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87f13d-54e3-47b2-89ec-c6d0918582b8 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:33.828301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6048902c-9365-485d-83a2-124c50c57948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f11e023666434a619b4fe94724ace8f2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:33.850448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 994/1972] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:33 2022] DELETE /identity/v3/users/f11e023666434a619b4fe94724ace8f2 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:33.856676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d70bb3-2185-46b7-8c71-15841b1cf2e4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:33.857257 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d70bb3-2185-46b7-8c71-15841b1cf2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:33.857648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d70bb3-2185-46b7-8c71-15841b1cf2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:33.863544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 979/1973] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:33.907162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d9d70bb3-2185-46b7-8c71-15841b1cf2e4 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:33.909431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d9d70bb3-2185-46b7-8c71-15841b1cf2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:34.072485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 995/1974] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:34.082902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87f13d-54e3-47b2-89ec-c6d0918582b8 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:34.096783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87f13d-54e3-47b2-89ec-c6d0918582b8 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:34.161200 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4e64b-6e03-414d-92c5-d94e498c442a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:34.161923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4e64b-6e03-414d-92c5-d94e498c442a service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:34.162201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4e64b-6e03-414d-92c5-d94e498c442a service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:34.162340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4e64b-6e03-414d-92c5-d94e498c442a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:34.172728 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4e64b-6e03-414d-92c5-d94e498c442a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:34.174194 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4e64b-6e03-414d-92c5-d94e498c442a service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:34.186454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 980/1975] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:34.296098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d70bb3-2185-46b7-8c71-15841b1cf2e4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:34.297871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d9d70bb3-2185-46b7-8c71-15841b1cf2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:34.312836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d70bb3-2185-46b7-8c71-15841b1cf2e4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:34.318622 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:34.319453 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:34.319731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:34.320006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] PATH_INFO: `/v3/projects/469c94da6b974013a0a546d254e7f160` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:34.321336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=469c94da6b974013a0a546d254e7f160)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:34.323807 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:34.392531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '469c94da6b974013a0a546d254e7f160', 'request_id': 'req-4acb73c7-78d3-46fe-86f5-2321fb728e11'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:34.399077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '469c94da6b974013a0a546d254e7f160', 'request_id': 'req-4acb73c7-78d3-46fe-86f5-2321fb728e11'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:34.405584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '469c94da6b974013a0a546d254e7f160', 'request_id': 'req-4acb73c7-78d3-46fe-86f5-2321fb728e11'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:34.422483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] The token cache is being invalidate because project 469c94da6b974013a0a546d254e7f160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:34.422675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:34.423411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4acb73c7-78d3-46fe-86f5-2321fb728e11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:34.425428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 996/1976] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:34 2022] DELETE /identity/v3/projects/469c94da6b974013a0a546d254e7f160 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:34.431145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c73816-641b-4197-afdf-4f5d10cad7b9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:34.431835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c73816-641b-4197-afdf-4f5d10cad7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:34.432952 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c73816-641b-4197-afdf-4f5d10cad7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:34.456467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-16c73816-641b-4197-afdf-4f5d10cad7b9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:34.457976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-16c73816-641b-4197-afdf-4f5d10cad7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:34.506371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:34.528383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 981/1977] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:34.536285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a6206-4c63-41bf-b9df-b648ea55a5ee None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:34.536595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a6206-4c63-41bf-b9df-b648ea55a5ee None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:34.536848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a6206-4c63-41bf-b9df-b648ea55a5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:34.561621 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-090a6206-4c63-41bf-b9df-b648ea55a5ee None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:34.562863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-090a6206-4c63-41bf-b9df-b648ea55a5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:34.566098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c73816-641b-4197-afdf-4f5d10cad7b9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:34.567576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-16c73816-641b-4197-afdf-4f5d10cad7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:34.576464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 997/1978] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:34.583239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a6206-4c63-41bf-b9df-b648ea55a5ee None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:34.590223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c73816-641b-4197-afdf-4f5d10cad7b9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:34.599272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:34.600299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:34.600751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:34.600944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a6206-4c63-41bf-b9df-b648ea55a5ee None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:34.601198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] PATH_INFO: `/v3/projects/66cfd185e94645078db1df52c047cfae` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:34.604650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66cfd185e94645078db1df52c047cfae)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:34.608331 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5e02f-4c37-40da-9b4a-d8fbe5064536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:34.608536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:34.609271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5e02f-4c37-40da-9b4a-d8fbe5064536 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:34.609714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5e02f-4c37-40da-9b4a-d8fbe5064536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:34.610155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5e02f-4c37-40da-9b4a-d8fbe5064536 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:34.611243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5e02f-4c37-40da-9b4a-d8fbe5064536 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:34.611905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5e02f-4c37-40da-9b4a-d8fbe5064536 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:34.614327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5e02f-4c37-40da-9b4a-d8fbe5064536 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:34.638556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 998/1979] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:08:34.643470 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5e02f-4c37-40da-9b4a-d8fbe5064536 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:34.649271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:34.658002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5e02f-4c37-40da-9b4a-d8fbe5064536 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:34.665111 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77501b2c-006e-4d05-b38a-f4a2d7bfbd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:34.666937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77501b2c-006e-4d05-b38a-f4a2d7bfbd9a admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:34.667740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77501b2c-006e-4d05-b38a-f4a2d7bfbd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:34.668379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77501b2c-006e-4d05-b38a-f4a2d7bfbd9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:34.670145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77501b2c-006e-4d05-b38a-f4a2d7bfbd9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:34.673186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77501b2c-006e-4d05-b38a-f4a2d7bfbd9a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:34.695765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66cfd185e94645078db1df52c047cfae', 'request_id': 'req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:34.707735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66cfd185e94645078db1df52c047cfae', 'request_id': 'req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:34.716219 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 999/1980] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:34 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:34.720284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77501b2c-006e-4d05-b38a-f4a2d7bfbd9a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:34.725195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66cfd185e94645078db1df52c047cfae', 'request_id': 'req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:34.753730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] The token cache is being invalidate because project 66cfd185e94645078db1df52c047cfae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:34.754337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:34.755524 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:34.758259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 982/1981] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:34 2022] DELETE /identity/v3/projects/66cfd185e94645078db1df52c047cfae => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:34.764528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:34.768905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:34.809061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77501b2c-006e-4d05-b38a-f4a2d7bfbd9a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:34.817578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e2727-a5ab-4f23-98ba-44631efd5576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:34.818226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e2727-a5ab-4f23-98ba-44631efd5576 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:34.818554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e2727-a5ab-4f23-98ba-44631efd5576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:34.818777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e2727-a5ab-4f23-98ba-44631efd5576 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:34.819479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e2727-a5ab-4f23-98ba-44631efd5576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:34.821436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e2727-a5ab-4f23-98ba-44631efd5576 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:34.863224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1000/1982] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:34 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:34.867423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e2727-a5ab-4f23-98ba-44631efd5576 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:34.871041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e11e2727-a5ab-4f23-98ba-44631efd5576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:34.874939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73ae6f-755a-4950-b9ce-94cb1307bfd6 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:34.891739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e2727-a5ab-4f23-98ba-44631efd5576 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:34.897223 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9bce81-670a-406f-9af3-617a14236117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:34.897856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bce81-670a-406f-9af3-617a14236117 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:34.898133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bce81-670a-406f-9af3-617a14236117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:34.898396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bce81-670a-406f-9af3-617a14236117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:34.899084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bce81-670a-406f-9af3-617a14236117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:34.899215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06df1ce-fe7d-4dde-8da4-b23c1555f1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:34.899882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06df1ce-fe7d-4dde-8da4-b23c1555f1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:34.900163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06df1ce-fe7d-4dde-8da4-b23c1555f1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:34.900442 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06df1ce-fe7d-4dde-8da4-b23c1555f1ec admin admin] PATH_INFO: `/v3/users/e73c44a18c774f76bb8768e7f1e336db` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:34.900909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bce81-670a-406f-9af3-617a14236117 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:34.901894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06df1ce-fe7d-4dde-8da4-b23c1555f1ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=e73c44a18c774f76bb8768e7f1e336db)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:34.904167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06df1ce-fe7d-4dde-8da4-b23c1555f1ec admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:34.913219 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1001/1983] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:08:34.918418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9bce81-670a-406f-9af3-617a14236117 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:34.946215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9bce81-670a-406f-9af3-617a14236117 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:34.959804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e0ed0-bfc9-4251-baf0-0b481f981a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:34.960517 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e0ed0-bfc9-4251-baf0-0b481f981a75 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:34.960911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e0ed0-bfc9-4251-baf0-0b481f981a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:34.961204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e0ed0-bfc9-4251-baf0-0b481f981a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:34.961922 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e0ed0-bfc9-4251-baf0-0b481f981a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:34.963696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e0ed0-bfc9-4251-baf0-0b481f981a75 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:35.000980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1002/1984] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:35.005239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e0ed0-bfc9-4251-baf0-0b481f981a75 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:35.006118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c06df1ce-fe7d-4dde-8da4-b23c1555f1ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e73c44a18c774f76bb8768e7f1e336db'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:35.025624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e0ed0-bfc9-4251-baf0-0b481f981a75 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:35.029811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c06df1ce-fe7d-4dde-8da4-b23c1555f1ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e73c44a18c774f76bb8768e7f1e336db'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:35.037597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:35.042702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e25062-0223-4dec-b4d1-417e73abfa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:35.043363 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e25062-0223-4dec-b4d1-417e73abfa75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:35.043725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e25062-0223-4dec-b4d1-417e73abfa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:35.044611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e25062-0223-4dec-b4d1-417e73abfa75 admin admin] PATH_INFO: `/v3/projects/78cce6768e1a473f9056d179b605df6d/users/4386cce77ec047f480e9999aeb103da5/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:35.050066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c06df1ce-fe7d-4dde-8da4-b23c1555f1ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e73c44a18c774f76bb8768e7f1e336db'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:35.053068 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c06df1ce-fe7d-4dde-8da4-b23c1555f1ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e73c44a18c774f76bb8768e7f1e336db'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:35.066586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c06df1ce-fe7d-4dde-8da4-b23c1555f1ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e73c44a18c774f76bb8768e7f1e336db'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:35.081501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e25062-0223-4dec-b4d1-417e73abfa75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78cce6768e1a473f9056d179b605df6d, user_id=4386cce77ec047f480e9999aeb103da5, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:35.084272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e25062-0223-4dec-b4d1-417e73abfa75 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:35.091627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 983/1985] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:34 2022] DELETE /identity/v3/users/e73c44a18c774f76bb8768e7f1e336db => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:35.105517 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1003/1986] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:35 2022] PUT /identity/v3/projects/78cce6768e1a473f9056d179b605df6d/users/4386cce77ec047f480e9999aeb103da5/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:35.110470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae9134-1dfe-473a-9a22-616eecff393f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:35.110755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae9134-1dfe-473a-9a22-616eecff393f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:35.111076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae9134-1dfe-473a-9a22-616eecff393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:35.146861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6eae9134-1dfe-473a-9a22-616eecff393f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:35.148987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6eae9134-1dfe-473a-9a22-616eecff393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:35.277542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 984/1987] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:35.300881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e25062-0223-4dec-b4d1-417e73abfa75 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:35.303987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-28e25062-0223-4dec-b4d1-417e73abfa75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:35.404115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e25062-0223-4dec-b4d1-417e73abfa75 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:35.408226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8646f9b-b956-441e-908d-cccbc9f2a3d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:35.408885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8646f9b-b956-441e-908d-cccbc9f2a3d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:35.409165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8646f9b-b956-441e-908d-cccbc9f2a3d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:35.409440 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8646f9b-b956-441e-908d-cccbc9f2a3d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:35.422704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8646f9b-b956-441e-908d-cccbc9f2a3d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:35.424200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8646f9b-b956-441e-908d-cccbc9f2a3d2 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:35.436273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1004/1988] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:35.505324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae9134-1dfe-473a-9a22-616eecff393f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:35.507075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6eae9134-1dfe-473a-9a22-616eecff393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:35.528075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae9134-1dfe-473a-9a22-616eecff393f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:35.534934 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:35.536018 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:35.536369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:35.536701 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] PATH_INFO: `/v3/projects/335600a391044a94a709fc70cce59278` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:35.537892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=335600a391044a94a709fc70cce59278)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:35.540138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:35.611210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '335600a391044a94a709fc70cce59278', 'request_id': 'req-4bb64c27-5480-4334-a74b-20e7ee44c0c3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:35.617295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '335600a391044a94a709fc70cce59278', 'request_id': 'req-4bb64c27-5480-4334-a74b-20e7ee44c0c3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:35.624503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '335600a391044a94a709fc70cce59278', 'request_id': 'req-4bb64c27-5480-4334-a74b-20e7ee44c0c3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:35.639544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] The token cache is being invalidate because project 335600a391044a94a709fc70cce59278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:35.639859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:35.641265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4bb64c27-5480-4334-a74b-20e7ee44c0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:35.643492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 985/1989] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:35 2022] DELETE /identity/v3/projects/335600a391044a94a709fc70cce59278 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:35.650137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5c05c-99d7-42c0-a0fb-1c95a2a68ea9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:35.650853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5c05c-99d7-42c0-a0fb-1c95a2a68ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:35.651402 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5c05c-99d7-42c0-a0fb-1c95a2a68ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:35.688204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-eca5c05c-99d7-42c0-a0fb-1c95a2a68ea9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:35.690319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-eca5c05c-99d7-42c0-a0fb-1c95a2a68ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:35.730806 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:35.809412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1005/1990] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:35.819176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78324f3f-15de-4b88-b414-5b2697b07568 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:35.819462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78324f3f-15de-4b88-b414-5b2697b07568 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:35.819731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78324f3f-15de-4b88-b414-5b2697b07568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:35.842284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-78324f3f-15de-4b88-b414-5b2697b07568 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:35.843524 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-78324f3f-15de-4b88-b414-5b2697b07568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:35.854804 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 986/1991] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:35.857497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5c05c-99d7-42c0-a0fb-1c95a2a68ea9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:35.869760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5c05c-99d7-42c0-a0fb-1c95a2a68ea9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:35.876273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717823a-8f75-4be4-bbd8-8574dfe48c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:35.876964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717823a-8f75-4be4-bbd8-8574dfe48c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:35.877241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717823a-8f75-4be4-bbd8-8574dfe48c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:35.877535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717823a-8f75-4be4-bbd8-8574dfe48c4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:35.878220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717823a-8f75-4be4-bbd8-8574dfe48c4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:35.878488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717823a-8f75-4be4-bbd8-8574dfe48c4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:35.886250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717823a-8f75-4be4-bbd8-8574dfe48c4c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:35.905425 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1006/1992] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:08:35.907012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78324f3f-15de-4b88-b414-5b2697b07568 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:35.918393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78324f3f-15de-4b88-b414-5b2697b07568 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:35.923659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a167a6-2d5b-474e-b7d4-f3f461c58bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:35.925596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a167a6-2d5b-474e-b7d4-f3f461c58bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:35.925596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a167a6-2d5b-474e-b7d4-f3f461c58bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:35.925596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a167a6-2d5b-474e-b7d4-f3f461c58bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:35.926167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a167a6-2d5b-474e-b7d4-f3f461c58bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:35.928116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a167a6-2d5b-474e-b7d4-f3f461c58bf3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:35.980697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 987/1993] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:35 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:35.984905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717823a-8f75-4be4-bbd8-8574dfe48c4c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:35.997361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717823a-8f75-4be4-bbd8-8574dfe48c4c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:36.073525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2ddb5-7129-43d7-bd45-85ff28c0beb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:36.074067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2ddb5-7129-43d7-bd45-85ff28c0beb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:36.074343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2ddb5-7129-43d7-bd45-85ff28c0beb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:36.074616 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2ddb5-7129-43d7-bd45-85ff28c0beb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:36.075304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2ddb5-7129-43d7-bd45-85ff28c0beb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:36.077316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2ddb5-7129-43d7-bd45-85ff28c0beb4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:36.099803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1007/1994] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:35 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:36.103996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a167a6-2d5b-474e-b7d4-f3f461c58bf3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:36.114993 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a167a6-2d5b-474e-b7d4-f3f461c58bf3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:36.122304 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210bd3f-e1ac-46f3-972a-6e5976eee52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:36.123068 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210bd3f-e1ac-46f3-972a-6e5976eee52b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:36.125070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210bd3f-e1ac-46f3-972a-6e5976eee52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:36.125070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210bd3f-e1ac-46f3-972a-6e5976eee52b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:36.125070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210bd3f-e1ac-46f3-972a-6e5976eee52b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:36.126281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210bd3f-e1ac-46f3-972a-6e5976eee52b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:36.143441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2ddb5-7129-43d7-bd45-85ff28c0beb4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:36.147128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 988/1995] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:36.154741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2ddb5-7129-43d7-bd45-85ff28c0beb4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:36.160080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169f377-eee5-48f3-a6a7-e7a767c0fbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:36.160711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f377-eee5-48f3-a6a7-e7a767c0fbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:36.160994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f377-eee5-48f3-a6a7-e7a767c0fbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:36.161283 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f377-eee5-48f3-a6a7-e7a767c0fbf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:36.162173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169f377-eee5-48f3-a6a7-e7a767c0fbf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:36.164197 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169f377-eee5-48f3-a6a7-e7a767c0fbf7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:36.176897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1008/1996] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:36.181931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210bd3f-e1ac-46f3-972a-6e5976eee52b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:36.209631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210bd3f-e1ac-46f3-972a-6e5976eee52b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:36.230799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405f479-00cc-4600-b034-3e9260e8d665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:36.232267 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405f479-00cc-4600-b034-3e9260e8d665 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:36.233595 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405f479-00cc-4600-b034-3e9260e8d665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:36.234085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405f479-00cc-4600-b034-3e9260e8d665 admin admin] PATH_INFO: `/v3/projects/926a18ba6a0e4f7394798b27a35e601b/users/029fda9fc36c471080a06b497632ac7f/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:36.263858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405f479-00cc-4600-b034-3e9260e8d665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=926a18ba6a0e4f7394798b27a35e601b, user_id=029fda9fc36c471080a06b497632ac7f, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:36.267449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405f479-00cc-4600-b034-3e9260e8d665 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:36.291010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 989/1997] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:36 2022] PUT /identity/v3/projects/926a18ba6a0e4f7394798b27a35e601b/users/029fda9fc36c471080a06b497632ac7f/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:36.296030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13880d3-0d38-49c6-af8c-3c4f09857cd3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:36.296251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13880d3-0d38-49c6-af8c-3c4f09857cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:36.296557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13880d3-0d38-49c6-af8c-3c4f09857cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:36.328015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a13880d3-0d38-49c6-af8c-3c4f09857cd3 None None] MFA Rules not processed for user `029fda9fc36c471080a06b497632ac7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:36.329669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a13880d3-0d38-49c6-af8c-3c4f09857cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:36.427251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1009/1998] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:36.428439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405f479-00cc-4600-b034-3e9260e8d665 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:36.447201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:36.527826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405f479-00cc-4600-b034-3e9260e8d665 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:36.534336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d029048-0e00-4f32-9926-bb5c411e7b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:36.535182 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029048-0e00-4f32-9926-bb5c411e7b3d admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:36.535561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029048-0e00-4f32-9926-bb5c411e7b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:36.535945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029048-0e00-4f32-9926-bb5c411e7b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:36.536893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d029048-0e00-4f32-9926-bb5c411e7b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:36.539460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d029048-0e00-4f32-9926-bb5c411e7b3d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:36.577147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 990/1999] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:36 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:36.591940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13880d3-0d38-49c6-af8c-3c4f09857cd3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:36.615251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13880d3-0d38-49c6-af8c-3c4f09857cd3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:36.648695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:36.709271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b388b-a59e-4ea2-b208-ffddd646bcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:36.710082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b388b-a59e-4ea2-b208-ffddd646bcf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:36.710364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b388b-a59e-4ea2-b208-ffddd646bcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:36.710638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b388b-a59e-4ea2-b208-ffddd646bcf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:36.711345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b388b-a59e-4ea2-b208-ffddd646bcf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:36.713344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b388b-a59e-4ea2-b208-ffddd646bcf8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:36.757751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1010/2000] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:36 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:36.763255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d029048-0e00-4f32-9926-bb5c411e7b3d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:36.797118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d029048-0e00-4f32-9926-bb5c411e7b3d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:36.809361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d48e96-3e2c-475e-b499-a7fdce3dee69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:36.810699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d48e96-3e2c-475e-b499-a7fdce3dee69 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:36.811191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d48e96-3e2c-475e-b499-a7fdce3dee69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:36.812986 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d48e96-3e2c-475e-b499-a7fdce3dee69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:36.813967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d48e96-3e2c-475e-b499-a7fdce3dee69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:36.816007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d48e96-3e2c-475e-b499-a7fdce3dee69 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:36.831290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 991/2001] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:36.835503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b388b-a59e-4ea2-b208-ffddd646bcf8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:36.846321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b388b-a59e-4ea2-b208-ffddd646bcf8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:36.851116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c65a74-f149-42dd-8ea6-714e6934dc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:36.851947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c65a74-f149-42dd-8ea6-714e6934dc5a admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:36.852228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c65a74-f149-42dd-8ea6-714e6934dc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:36.852499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c65a74-f149-42dd-8ea6-714e6934dc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:36.853138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c65a74-f149-42dd-8ea6-714e6934dc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:36.855345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c65a74-f149-42dd-8ea6-714e6934dc5a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:36.866842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1011/2002] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:36.872014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d48e96-3e2c-475e-b499-a7fdce3dee69 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:36.883066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d48e96-3e2c-475e-b499-a7fdce3dee69 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:36.889533 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756084a1-1a8f-4fb3-ab92-64f303c88c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:36.890291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756084a1-1a8f-4fb3-ab92-64f303c88c64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:36.890643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756084a1-1a8f-4fb3-ab92-64f303c88c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:36.891023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756084a1-1a8f-4fb3-ab92-64f303c88c64 admin admin] PATH_INFO: `/v3/projects/14d4e6a9eb3245b184b5d0b66e549e60/users/5e5044982253451eb983cf8b5ee44c15/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:36.911766 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756084a1-1a8f-4fb3-ab92-64f303c88c64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d4e6a9eb3245b184b5d0b66e549e60, user_id=5e5044982253451eb983cf8b5ee44c15, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:36.917262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756084a1-1a8f-4fb3-ab92-64f303c88c64 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:36.947094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 992/2003] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:36 2022] PUT /identity/v3/projects/14d4e6a9eb3245b184b5d0b66e549e60/users/5e5044982253451eb983cf8b5ee44c15/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:36.957857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a823e3e-edea-4fb5-b413-c6a4260bbf1a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:36.958711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a823e3e-edea-4fb5-b413-c6a4260bbf1a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:36.959023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a823e3e-edea-4fb5-b413-c6a4260bbf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:37.025245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8a823e3e-edea-4fb5-b413-c6a4260bbf1a None None] MFA Rules not processed for user `5e5044982253451eb983cf8b5ee44c15`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:37.027989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8a823e3e-edea-4fb5-b413-c6a4260bbf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:37.100026 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95144fe-6e13-4eda-98d0-0b75fcb38223 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:37.100556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95144fe-6e13-4eda-98d0-0b75fcb38223 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:37.100873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95144fe-6e13-4eda-98d0-0b75fcb38223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:37.127291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1012/2004] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:37.131485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a823e3e-edea-4fb5-b413-c6a4260bbf1a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:37.136303 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b95144fe-6e13-4eda-98d0-0b75fcb38223 None None] MFA Rules not processed for user `4386cce77ec047f480e9999aeb103da5`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:37.139650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b95144fe-6e13-4eda-98d0-0b75fcb38223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:37.238181 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a823e3e-edea-4fb5-b413-c6a4260bbf1a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:37.243642 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e30262-c9ff-4359-9035-a11867f9ebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:37.244244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e30262-c9ff-4359-9035-a11867f9ebc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:37.244681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e30262-c9ff-4359-9035-a11867f9ebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:37.245061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e30262-c9ff-4359-9035-a11867f9ebc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:37.245896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e30262-c9ff-4359-9035-a11867f9ebc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:37.248689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e30262-c9ff-4359-9035-a11867f9ebc7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:37.297939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1013/2005] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:37 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:37.304031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e30262-c9ff-4359-9035-a11867f9ebc7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:37.334913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:37.362742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 993/2006] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:37.376656 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95144fe-6e13-4eda-98d0-0b75fcb38223 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:37.378122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b95144fe-6e13-4eda-98d0-0b75fcb38223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:37.463202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:37.466154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e30262-c9ff-4359-9035-a11867f9ebc7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:37.472003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27a4e8-6658-4171-ac8d-a24d95c5a0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:37.472750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27a4e8-6658-4171-ac8d-a24d95c5a0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:37.473098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27a4e8-6658-4171-ac8d-a24d95c5a0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:37.473381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27a4e8-6658-4171-ac8d-a24d95c5a0b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:37.474032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27a4e8-6658-4171-ac8d-a24d95c5a0b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:37.475522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95144fe-6e13-4eda-98d0-0b75fcb38223 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:37.476036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27a4e8-6658-4171-ac8d-a24d95c5a0b9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:37.504931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1014/2007] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:37 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:37.509295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27a4e8-6658-4171-ac8d-a24d95c5a0b9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:37.542632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27a4e8-6658-4171-ac8d-a24d95c5a0b9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:37.553931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72fc4c-574a-4b4c-9815-f2cc60868ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:37.570306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72fc4c-574a-4b4c-9815-f2cc60868ada admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:37.571044 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72fc4c-574a-4b4c-9815-f2cc60868ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:37.571828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72fc4c-574a-4b4c-9815-f2cc60868ada admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:37.573132 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72fc4c-574a-4b4c-9815-f2cc60868ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:37.574811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72fc4c-574a-4b4c-9815-f2cc60868ada admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:37.589563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1015/2008] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:37.592952 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72fc4c-574a-4b4c-9815-f2cc60868ada admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:37.595180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c54fa-5de2-4b98-a29a-79483240f33c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:37.596028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c54fa-5de2-4b98-a29a-79483240f33c service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:37.596186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c54fa-5de2-4b98-a29a-79483240f33c service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:37.596694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c54fa-5de2-4b98-a29a-79483240f33c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:37.606627 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72fc4c-574a-4b4c-9815-f2cc60868ada admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:37.615976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb96ad-68e1-459e-ad04-16a02fe551cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:37.617217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb96ad-68e1-459e-ad04-16a02fe551cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:37.617727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb96ad-68e1-459e-ad04-16a02fe551cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:37.618266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb96ad-68e1-459e-ad04-16a02fe551cc admin admin] PATH_INFO: `/v3/projects/3223775929a642b18e5fabff522e4bc6/users/724cf89f6df2444f98bda21674152c9e/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:37.653668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb96ad-68e1-459e-ad04-16a02fe551cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3223775929a642b18e5fabff522e4bc6, user_id=724cf89f6df2444f98bda21674152c9e, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:37.659473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb96ad-68e1-459e-ad04-16a02fe551cc admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:37.682651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c54fa-5de2-4b98-a29a-79483240f33c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:37.690082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c54fa-5de2-4b98-a29a-79483240f33c service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:37.690789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1016/2009] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:37 2022] PUT /identity/v3/projects/3223775929a642b18e5fabff522e4bc6/users/724cf89f6df2444f98bda21674152c9e/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:37.693892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb96ad-68e1-459e-ad04-16a02fe551cc admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:37.715800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 994/2010] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:37.840535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb96ad-68e1-459e-ad04-16a02fe551cc admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:37.847138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913de82-8c44-48af-a779-5980b1558db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:37.848007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913de82-8c44-48af-a779-5980b1558db6 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:37.848381 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913de82-8c44-48af-a779-5980b1558db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:37.848743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913de82-8c44-48af-a779-5980b1558db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:37.849601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913de82-8c44-48af-a779-5980b1558db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:37.851242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913de82-8c44-48af-a779-5980b1558db6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:37.872917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c54fa-5de2-4b98-a29a-79483240f33c service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:37.885548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1017/2011] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:08:37.888337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c54fa-5de2-4b98-a29a-79483240f33c service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:37.895482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b620a94-284f-44d0-8bc4-e856b93db143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:37.896168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b620a94-284f-44d0-8bc4-e856b93db143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:37.896447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b620a94-284f-44d0-8bc4-e856b93db143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:37.896721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b620a94-284f-44d0-8bc4-e856b93db143 admin admin] PATH_INFO: `/v3/projects/3223775929a642b18e5fabff522e4bc6/users/724cf89f6df2444f98bda21674152c9e/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:37.899751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b620a94-284f-44d0-8bc4-e856b93db143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3223775929a642b18e5fabff522e4bc6, user_id=724cf89f6df2444f98bda21674152c9e, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:37.905161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b620a94-284f-44d0-8bc4-e856b93db143 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:37.937832 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1594613-f7f3-4376-986f-23c937930a98 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:37.938122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1594613-f7f3-4376-986f-23c937930a98 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:37.938395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1594613-f7f3-4376-986f-23c937930a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:37.938502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 995/2012] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:37 2022] PUT /identity/v3/projects/3223775929a642b18e5fabff522e4bc6/users/724cf89f6df2444f98bda21674152c9e/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:38.001297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b1594613-f7f3-4376-986f-23c937930a98 None None] MFA Rules not processed for user `724cf89f6df2444f98bda21674152c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:38.002858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b1594613-f7f3-4376-986f-23c937930a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:38.142970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:38.206168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1018/2013] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:38.364144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b620a94-284f-44d0-8bc4-e856b93db143 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:38.454773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b620a94-284f-44d0-8bc4-e856b93db143 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:38.460128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1afaf-1fae-43f1-aacf-f8163724b15f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:38.461188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1afaf-1fae-43f1-aacf-f8163724b15f service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:38.461483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1afaf-1fae-43f1-aacf-f8163724b15f service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:38.461759 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1afaf-1fae-43f1-aacf-f8163724b15f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:38.475144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1afaf-1fae-43f1-aacf-f8163724b15f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:38.476844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1afaf-1fae-43f1-aacf-f8163724b15f service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:38.488418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 996/2014] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:38.634884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1594613-f7f3-4376-986f-23c937930a98 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:38.645031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1594613-f7f3-4376-986f-23c937930a98 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:38.656145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeff04f-b5d7-4625-93fa-b829ca3217aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:38.656890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeff04f-b5d7-4625-93fa-b829ca3217aa service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:38.657304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeff04f-b5d7-4625-93fa-b829ca3217aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:38.657670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeff04f-b5d7-4625-93fa-b829ca3217aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:38.740527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeff04f-b5d7-4625-93fa-b829ca3217aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:38.742367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeff04f-b5d7-4625-93fa-b829ca3217aa service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:38.763402 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1019/2015] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:39.711820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dafcb-7ca3-4bcf-90c0-a5ee215bdcd6 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:39.712395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dafcb-7ca3-4bcf-90c0-a5ee215bdcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:39.712597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dafcb-7ca3-4bcf-90c0-a5ee215bdcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:39.733432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b17dafcb-7ca3-4bcf-90c0-a5ee215bdcd6 None None] MFA Rules not processed for user `4386cce77ec047f480e9999aeb103da5`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:39.734709 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b17dafcb-7ca3-4bcf-90c0-a5ee215bdcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:39.807694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:39.831123 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 997/2016] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:41.783026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328719b-70b1-4b05-accc-a4a1d7fa272f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:41.785352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328719b-70b1-4b05-accc-a4a1d7fa272f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:41.786419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328719b-70b1-4b05-accc-a4a1d7fa272f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:41.790542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1020/2017] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 17:08:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:08:41.798403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5783318-6798-4f14-9f77-54e6ff453947 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:41.798891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5783318-6798-4f14-9f77-54e6ff453947 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:41.799268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5783318-6798-4f14-9f77-54e6ff453947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:41.851088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a5783318-6798-4f14-9f77-54e6ff453947 None None] MFA Rules not processed for user `81088a327fd448ad8fec555f3d914be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:41.853013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a5783318-6798-4f14-9f77-54e6ff453947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:41.956940 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 998/2018] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 17:08:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:41.972754 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328719b-70b1-4b05-accc-a4a1d7fa272f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:41.976130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9328719b-70b1-4b05-accc-a4a1d7fa272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:42.004373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328719b-70b1-4b05-accc-a4a1d7fa272f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:42.010374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad641e-636d-456d-86ab-2cd249da88eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:42.011216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad641e-636d-456d-86ab-2cd249da88eb service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:42.011776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad641e-636d-456d-86ab-2cd249da88eb service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:42.012198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad641e-636d-456d-86ab-2cd249da88eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:42.029589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad641e-636d-456d-86ab-2cd249da88eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:42.031460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad641e-636d-456d-86ab-2cd249da88eb service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:42.044296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1021/2019] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:42.457873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5783318-6798-4f14-9f77-54e6ff453947 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:42.560022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5783318-6798-4f14-9f77-54e6ff453947 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:42.575336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49255da4-90bf-455b-bc7a-21693f0d8ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:42.576316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49255da4-90bf-455b-bc7a-21693f0d8ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:42.576910 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49255da4-90bf-455b-bc7a-21693f0d8ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:42.577622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49255da4-90bf-455b-bc7a-21693f0d8ca6 admin admin] PATH_INFO: `/v3/users/029fda9fc36c471080a06b497632ac7f` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:42.580227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49255da4-90bf-455b-bc7a-21693f0d8ca6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=029fda9fc36c471080a06b497632ac7f)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:42.584252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49255da4-90bf-455b-bc7a-21693f0d8ca6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:42.679873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-49255da4-90bf-455b-bc7a-21693f0d8ca6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '029fda9fc36c471080a06b497632ac7f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:42.705981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-49255da4-90bf-455b-bc7a-21693f0d8ca6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '029fda9fc36c471080a06b497632ac7f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:42.716735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-49255da4-90bf-455b-bc7a-21693f0d8ca6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '029fda9fc36c471080a06b497632ac7f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:42.717611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-49255da4-90bf-455b-bc7a-21693f0d8ca6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '029fda9fc36c471080a06b497632ac7f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:42.728432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-49255da4-90bf-455b-bc7a-21693f0d8ca6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '029fda9fc36c471080a06b497632ac7f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:42.738764 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 999/2020] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:42 2022] DELETE /identity/v3/users/029fda9fc36c471080a06b497632ac7f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:42.744354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f048d-2728-4e29-b13f-f20b11252331 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:42.745356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f048d-2728-4e29-b13f-f20b11252331 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:42.745745 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f048d-2728-4e29-b13f-f20b11252331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:42.780258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c09f048d-2728-4e29-b13f-f20b11252331 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:42.781725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c09f048d-2728-4e29-b13f-f20b11252331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:42.892734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1022/2021] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:42.893772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb08e3-17bb-4a2f-86f6-4e97e91346b9 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:42.895172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb08e3-17bb-4a2f-86f6-4e97e91346b9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:42.898554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb08e3-17bb-4a2f-86f6-4e97e91346b9 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:42.902325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1000/2022] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 17:08:42 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:08:42.920137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f048d-2728-4e29-b13f-f20b11252331 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:42.922110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c09f048d-2728-4e29-b13f-f20b11252331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:42.931723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dc0db-6e21-42eb-bce9-9f85458e502d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:42.932228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dc0db-6e21-42eb-bce9-9f85458e502d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:42.934469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dc0db-6e21-42eb-bce9-9f85458e502d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:42.985802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ab4dc0db-6e21-42eb-bce9-9f85458e502d None None] MFA Rules not processed for user `81088a327fd448ad8fec555f3d914be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:42.988584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ab4dc0db-6e21-42eb-bce9-9f85458e502d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:43.000282 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:43.074175 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f048d-2728-4e29-b13f-f20b11252331 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:43.081673 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509ee12-38f7-4bd0-aacc-ad3d217f186b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:43.083359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509ee12-38f7-4bd0-aacc-ad3d217f186b service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:43.084079 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509ee12-38f7-4bd0-aacc-ad3d217f186b service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:43.084540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509ee12-38f7-4bd0-aacc-ad3d217f186b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:43.106516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509ee12-38f7-4bd0-aacc-ad3d217f186b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:43.109636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509ee12-38f7-4bd0-aacc-ad3d217f186b service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:43.140451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1023/2023] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:43.140451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509ee12-38f7-4bd0-aacc-ad3d217f186b service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:43.140451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1509ee12-38f7-4bd0-aacc-ad3d217f186b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:43.158983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1001/2024] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 17:08:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:43.159334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509ee12-38f7-4bd0-aacc-ad3d217f186b service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:43.160614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dc0db-6e21-42eb-bce9-9f85458e502d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:43.162167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ab4dc0db-6e21-42eb-bce9-9f85458e502d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:43.174736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:43.176137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:43.176490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:43.176836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] PATH_INFO: `/v3/users/f244c85df17e4a47aa6b63d032f19abb` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:43.177562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dc0db-6e21-42eb-bce9-9f85458e502d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:43.179064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f244c85df17e4a47aa6b63d032f19abb)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:43.180577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:43.185009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c593d0c-e6d3-4178-84ae-c428f886d0ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:43.186013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c593d0c-e6d3-4178-84ae-c428f886d0ec service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:43.186288 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c593d0c-e6d3-4178-84ae-c428f886d0ec service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:43.186562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c593d0c-e6d3-4178-84ae-c428f886d0ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:43.197095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c593d0c-e6d3-4178-84ae-c428f886d0ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:43.198080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:43.198892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c593d0c-e6d3-4178-84ae-c428f886d0ec service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:43.212073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1002/2025] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:08:43.267013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f244c85df17e4a47aa6b63d032f19abb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:43.284967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f244c85df17e4a47aa6b63d032f19abb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:43.295060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f244c85df17e4a47aa6b63d032f19abb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:43.307501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f244c85df17e4a47aa6b63d032f19abb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:43.308500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f244c85df17e4a47aa6b63d032f19abb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:43.316495 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1024/2026] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:43 2022] DELETE /identity/v3/users/f244c85df17e4a47aa6b63d032f19abb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:43.745847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c593d0c-e6d3-4178-84ae-c428f886d0ec service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:43.776576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c593d0c-e6d3-4178-84ae-c428f886d0ec service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:43.875271 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:43.875975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:43.876289 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:43.876625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] PATH_INFO: `/v3/projects/1fbebdb527ce42fca62d9da743135659` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:43.878126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fbebdb527ce42fca62d9da743135659)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:43.881011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:43.946537 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fbebdb527ce42fca62d9da743135659', 'request_id': 'req-985849ed-b47b-4ff6-a601-e44bfabafa52'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:43.961170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fbebdb527ce42fca62d9da743135659', 'request_id': 'req-985849ed-b47b-4ff6-a601-e44bfabafa52'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:43.973866 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fbebdb527ce42fca62d9da743135659', 'request_id': 'req-985849ed-b47b-4ff6-a601-e44bfabafa52'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:43.999308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] The token cache is being invalidate because project 1fbebdb527ce42fca62d9da743135659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:43.999923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.001774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.008176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1003/2027] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:43 2022] DELETE /identity/v3/projects/1fbebdb527ce42fca62d9da743135659 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:44.009052 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:44.011915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:44.031508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b98d52-5dd3-42bf-ae01-256bcf3fda40 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:44.122699 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:44.123588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:44.123987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:44.124360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] PATH_INFO: `/v3/users/b07f8d0748984ef5ad47422b3e73e467` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:44.125720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b07f8d0748984ef5ad47422b3e73e467)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:44.128068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:44.204838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b07f8d0748984ef5ad47422b3e73e467'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.215660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:44.217244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:44.233931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985849ed-b47b-4ff6-a601-e44bfabafa52 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:44.238574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b07f8d0748984ef5ad47422b3e73e467'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.240134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:44.241522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:44.241966 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:44.242393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] PATH_INFO: `/v3/projects/926a18ba6a0e4f7394798b27a35e601b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:44.243872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=926a18ba6a0e4f7394798b27a35e601b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:44.247951 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:44.255550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b07f8d0748984ef5ad47422b3e73e467'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.266448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b07f8d0748984ef5ad47422b3e73e467'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.269779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b07f8d0748984ef5ad47422b3e73e467'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.278836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:44.288869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1025/2028] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:44 2022] DELETE /identity/v3/users/b07f8d0748984ef5ad47422b3e73e467 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:44.324521 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '926a18ba6a0e4f7394798b27a35e601b', 'request_id': 'req-b7150607-3b7e-4435-8835-00bcb00bf090'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.346195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '926a18ba6a0e4f7394798b27a35e601b', 'request_id': 'req-b7150607-3b7e-4435-8835-00bcb00bf090'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.359916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '926a18ba6a0e4f7394798b27a35e601b', 'request_id': 'req-b7150607-3b7e-4435-8835-00bcb00bf090'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.380416 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] The token cache is being invalidate because project 926a18ba6a0e4f7394798b27a35e601b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:44.380846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.381711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.383750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1004/2029] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:44 2022] DELETE /identity/v3/projects/926a18ba6a0e4f7394798b27a35e601b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:44.388938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:44.390963 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:44.407468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:44.522937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b84127-b8f2-45a5-849e-945d98492d23 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:44.654327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78f6a2-cec5-4672-b431-99e7928f19d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:44.656643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78f6a2-cec5-4672-b431-99e7928f19d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:44.657467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78f6a2-cec5-4672-b431-99e7928f19d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:44.658239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78f6a2-cec5-4672-b431-99e7928f19d0 admin admin] PATH_INFO: `/v3/users/5e5044982253451eb983cf8b5ee44c15` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:44.660663 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78f6a2-cec5-4672-b431-99e7928f19d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e5044982253451eb983cf8b5ee44c15)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:44.665141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78f6a2-cec5-4672-b431-99e7928f19d0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:44.754370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ff78f6a2-cec5-4672-b431-99e7928f19d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e5044982253451eb983cf8b5ee44c15'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.784003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ff78f6a2-cec5-4672-b431-99e7928f19d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e5044982253451eb983cf8b5ee44c15'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.796245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ff78f6a2-cec5-4672-b431-99e7928f19d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e5044982253451eb983cf8b5ee44c15'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.807759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ff78f6a2-cec5-4672-b431-99e7928f19d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e5044982253451eb983cf8b5ee44c15'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.808490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ff78f6a2-cec5-4672-b431-99e7928f19d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e5044982253451eb983cf8b5ee44c15'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:44.819570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1026/2030] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:44 2022] DELETE /identity/v3/users/5e5044982253451eb983cf8b5ee44c15 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:44.897428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:44.900165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:44.920859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7150607-3b7e-4435-8835-00bcb00bf090 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:45.032097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:45.033549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:45.034150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:45.035278 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] PATH_INFO: `/v3/projects/568cf0f015a442fcbe5cd769fa775665` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:45.037114 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568cf0f015a442fcbe5cd769fa775665)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:45.042045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:45.126312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568cf0f015a442fcbe5cd769fa775665', 'request_id': 'req-d08640f0-1c36-4197-bcd0-daae40846f83'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:45.140276 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568cf0f015a442fcbe5cd769fa775665', 'request_id': 'req-d08640f0-1c36-4197-bcd0-daae40846f83'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:45.149700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568cf0f015a442fcbe5cd769fa775665', 'request_id': 'req-d08640f0-1c36-4197-bcd0-daae40846f83'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:45.167550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] The token cache is being invalidate because project 568cf0f015a442fcbe5cd769fa775665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:45.168269 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:45.169649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d08640f0-1c36-4197-bcd0-daae40846f83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:45.174714 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1005/2031] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:44 2022] DELETE /identity/v3/projects/568cf0f015a442fcbe5cd769fa775665 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:45.176741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8cd76-9102-43fe-9493-a79a061cc8a7 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:45.177249 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8cd76-9102-43fe-9493-a79a061cc8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:45.177379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8cd76-9102-43fe-9493-a79a061cc8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:45.200869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-75c8cd76-9102-43fe-9493-a79a061cc8a7 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:45.202931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-75c8cd76-9102-43fe-9493-a79a061cc8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:45.255940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:45.338709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1027/2032] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:45.340598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902c487-30ff-4e89-941f-1dd25f163990 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:45.340884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902c487-30ff-4e89-941f-1dd25f163990 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:45.341160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902c487-30ff-4e89-941f-1dd25f163990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:45.365588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9902c487-30ff-4e89-941f-1dd25f163990 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:45.367155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9902c487-30ff-4e89-941f-1dd25f163990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:45.382311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1006/2033] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:45.383206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8cd76-9102-43fe-9493-a79a061cc8a7 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:45.395615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8cd76-9102-43fe-9493-a79a061cc8a7 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:45.402123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bf40a-3802-4425-a8c1-558e085e5afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:45.403130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bf40a-3802-4425-a8c1-558e085e5afd admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:45.403584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bf40a-3802-4425-a8c1-558e085e5afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:45.404068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bf40a-3802-4425-a8c1-558e085e5afd admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:45.404944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261bf40a-3802-4425-a8c1-558e085e5afd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:45.405390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261bf40a-3802-4425-a8c1-558e085e5afd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:45.407601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261bf40a-3802-4425-a8c1-558e085e5afd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:45.437108 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1028/2034] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 17:08:45.456754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902c487-30ff-4e89-941f-1dd25f163990 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:45.467147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902c487-30ff-4e89-941f-1dd25f163990 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:45.473482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72970-daad-4d74-89a7-19395cff8681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:45.474459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72970-daad-4d74-89a7-19395cff8681 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:45.474820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72970-daad-4d74-89a7-19395cff8681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:45.475226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72970-daad-4d74-89a7-19395cff8681 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:45.476064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b72970-daad-4d74-89a7-19395cff8681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:45.479789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b72970-daad-4d74-89a7-19395cff8681 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:45.523045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1007/2035] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:45 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:45.523416 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bf40a-3802-4425-a8c1-558e085e5afd admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:45.546987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bf40a-3802-4425-a8c1-558e085e5afd admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:45.670325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72970-daad-4d74-89a7-19395cff8681 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:45.672326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-59b72970-daad-4d74-89a7-19395cff8681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:45.674672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45817423-92c4-4061-8d80-651cc6e87f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:45.677399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45817423-92c4-4061-8d80-651cc6e87f85 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:45.677908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45817423-92c4-4061-8d80-651cc6e87f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:45.678419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45817423-92c4-4061-8d80-651cc6e87f85 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:45.679652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45817423-92c4-4061-8d80-651cc6e87f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:45.683528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45817423-92c4-4061-8d80-651cc6e87f85 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:45.688742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72970-daad-4d74-89a7-19395cff8681 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:45.695318 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:45.696205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:45.696626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:45.696977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] PATH_INFO: `/v3/projects/14d4e6a9eb3245b184b5d0b66e549e60` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:45.698804 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d4e6a9eb3245b184b5d0b66e549e60)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:45.702057 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:45.704488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:45.751088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1029/2036] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:45 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:45.758496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45817423-92c4-4061-8d80-651cc6e87f85 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:45.783818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45817423-92c4-4061-8d80-651cc6e87f85 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:45.801064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c580ad5-ace7-4aa3-97a2-0dea4f1403e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:45.801551 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c580ad5-ace7-4aa3-97a2-0dea4f1403e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:45.802564 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c580ad5-ace7-4aa3-97a2-0dea4f1403e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:45.803144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c580ad5-ace7-4aa3-97a2-0dea4f1403e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:45.806049 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c580ad5-ace7-4aa3-97a2-0dea4f1403e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:45.809430 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c580ad5-ace7-4aa3-97a2-0dea4f1403e0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:45.842250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1030/2037] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:45.843388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d4e6a9eb3245b184b5d0b66e549e60', 'request_id': 'req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:45.846206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c580ad5-ace7-4aa3-97a2-0dea4f1403e0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:45.883126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d4e6a9eb3245b184b5d0b66e549e60', 'request_id': 'req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:45.903102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c580ad5-ace7-4aa3-97a2-0dea4f1403e0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:45.906809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d4e6a9eb3245b184b5d0b66e549e60', 'request_id': 'req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:45.935795 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] The token cache is being invalidate because project 14d4e6a9eb3245b184b5d0b66e549e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:45.937162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:45.938210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:45.946788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1008/2038] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:45 2022] DELETE /identity/v3/projects/14d4e6a9eb3245b184b5d0b66e549e60 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:45.948033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:45.951839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:46.088638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced4ddd-4cd6-4b8f-b0b4-62b3f8a1b4f2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:46.097514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:46.098383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:46.098767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:46.099372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] PATH_INFO: `/v3/users/724cf89f6df2444f98bda21674152c9e` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:46.101437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=724cf89f6df2444f98bda21674152c9e)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:46.104791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:46.120484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549948d-e8e3-41b7-98b1-800ee8ab71af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:46.121673 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549948d-e8e3-41b7-98b1-800ee8ab71af admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:46.122171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549948d-e8e3-41b7-98b1-800ee8ab71af admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:46.122655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549948d-e8e3-41b7-98b1-800ee8ab71af admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:46.123930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549948d-e8e3-41b7-98b1-800ee8ab71af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:46.131480 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549948d-e8e3-41b7-98b1-800ee8ab71af admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:46.163769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1031/2039] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:46.170027 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549948d-e8e3-41b7-98b1-800ee8ab71af admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:46.175786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1549948d-e8e3-41b7-98b1-800ee8ab71af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:46.216536 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549948d-e8e3-41b7-98b1-800ee8ab71af admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:46.223882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a917f-77a1-4111-a60a-bf059aa790e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:46.224479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a917f-77a1-4111-a60a-bf059aa790e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:46.224861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a917f-77a1-4111-a60a-bf059aa790e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:46.225286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a917f-77a1-4111-a60a-bf059aa790e6 admin admin] PATH_INFO: `/v3/projects/c59b0234992042d5acb002ebabc09da5/users/d2d572a9f71f488b9cb9d5254afbd512/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:46.234273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724cf89f6df2444f98bda21674152c9e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:46.266653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a917f-77a1-4111-a60a-bf059aa790e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59b0234992042d5acb002ebabc09da5, user_id=d2d572a9f71f488b9cb9d5254afbd512, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:46.270234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a917f-77a1-4111-a60a-bf059aa790e6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:46.277796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724cf89f6df2444f98bda21674152c9e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:46.299040 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724cf89f6df2444f98bda21674152c9e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:46.300031 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724cf89f6df2444f98bda21674152c9e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:46.302016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1032/2040] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:46 2022] PUT /identity/v3/projects/c59b0234992042d5acb002ebabc09da5/users/d2d572a9f71f488b9cb9d5254afbd512/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:46.315420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aebd3-de7e-4fae-9d85-151ed6ee69b9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:46.316115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aebd3-de7e-4fae-9d85-151ed6ee69b9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:46.316555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aebd3-de7e-4fae-9d85-151ed6ee69b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:46.325379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724cf89f6df2444f98bda21674152c9e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:46.338723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1009/2041] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:45 2022] DELETE /identity/v3/users/724cf89f6df2444f98bda21674152c9e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:46.361941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e76aebd3-de7e-4fae-9d85-151ed6ee69b9 None None] MFA Rules not processed for user `d2d572a9f71f488b9cb9d5254afbd512`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:46.363642 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e76aebd3-de7e-4fae-9d85-151ed6ee69b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:46.368691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:46.464712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1033/2042] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:46.472767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:46.475196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:46.506910 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:46.637246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe04df2-4c4f-4798-868c-f3c6ac3126d8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:46.643259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb316c-6954-400a-ab85-5eff527a7b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:46.644040 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb316c-6954-400a-ab85-5eff527a7b21 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:46.644404 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb316c-6954-400a-ab85-5eff527a7b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:46.644763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb316c-6954-400a-ab85-5eff527a7b21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:46.645520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb316c-6954-400a-ab85-5eff527a7b21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:46.647680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb316c-6954-400a-ab85-5eff527a7b21 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:46.687499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1010/2043] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:46 2022] POST /identity/v3/projects => generated 377 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:46.688974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76aebd3-de7e-4fae-9d85-151ed6ee69b9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:46.700844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76aebd3-de7e-4fae-9d85-151ed6ee69b9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:46.818427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be737e1-a94f-4ca4-8d7c-370f8a0c026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:46.819083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be737e1-a94f-4ca4-8d7c-370f8a0c026a admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:46.819368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be737e1-a94f-4ca4-8d7c-370f8a0c026a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:46.819642 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be737e1-a94f-4ca4-8d7c-370f8a0c026a admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:46.820359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be737e1-a94f-4ca4-8d7c-370f8a0c026a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:46.823348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be737e1-a94f-4ca4-8d7c-370f8a0c026a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:46.856851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1034/2044] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:46 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:46.864742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb316c-6954-400a-ab85-5eff527a7b21 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:46.885496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb316c-6954-400a-ab85-5eff527a7b21 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:46.890452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ab0eb-8441-4043-a881-5f18f3320a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:46.891178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab0eb-8441-4043-a881-5f18f3320a49 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:46.891465 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab0eb-8441-4043-a881-5f18f3320a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:46.891794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab0eb-8441-4043-a881-5f18f3320a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:46.892473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ab0eb-8441-4043-a881-5f18f3320a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:46.894106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ab0eb-8441-4043-a881-5f18f3320a49 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:46.907542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1011/2045] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:46.912090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be737e1-a94f-4ca4-8d7c-370f8a0c026a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:46.928240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be737e1-a94f-4ca4-8d7c-370f8a0c026a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:46.935420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972ce88-3046-4ccc-9f58-d77153a52fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:46.936313 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ce88-3046-4ccc-9f58-d77153a52fef admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:46.936681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ce88-3046-4ccc-9f58-d77153a52fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:46.937811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ce88-3046-4ccc-9f58-d77153a52fef admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:46.938021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972ce88-3046-4ccc-9f58-d77153a52fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:46.939849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972ce88-3046-4ccc-9f58-d77153a52fef admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:46.954524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1035/2046] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:08:46.959375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ab0eb-8441-4043-a881-5f18f3320a49 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:46.978246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ab0eb-8441-4043-a881-5f18f3320a49 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:46.985290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b360bf-59ed-4909-8f84-e3376cc2c675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:46.986541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b360bf-59ed-4909-8f84-e3376cc2c675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:46.986829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b360bf-59ed-4909-8f84-e3376cc2c675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:46.987199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b360bf-59ed-4909-8f84-e3376cc2c675 admin admin] PATH_INFO: `/v3/projects/1dd3e58a14ac4b0d847f8c21550fd5fa/users/39bb99aa285f4d9ab099f7c8f915449f/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.012040 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b360bf-59ed-4909-8f84-e3376cc2c675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd3e58a14ac4b0d847f8c21550fd5fa, user_id=39bb99aa285f4d9ab099f7c8f915449f, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:47.014986 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b360bf-59ed-4909-8f84-e3376cc2c675 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:47.038073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1012/2047] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:46 2022] PUT /identity/v3/projects/1dd3e58a14ac4b0d847f8c21550fd5fa/users/39bb99aa285f4d9ab099f7c8f915449f/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:47.049841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49522cea-c12b-472c-989d-70b048e69daa None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:47.050406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49522cea-c12b-472c-989d-70b048e69daa None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:47.050660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49522cea-c12b-472c-989d-70b048e69daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.086840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-49522cea-c12b-472c-989d-70b048e69daa None None] MFA Rules not processed for user `39bb99aa285f4d9ab099f7c8f915449f`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:47.087636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-49522cea-c12b-472c-989d-70b048e69daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:47.118754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b360bf-59ed-4909-8f84-e3376cc2c675 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:47.120801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d3b360bf-59ed-4909-8f84-e3376cc2c675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:47.243979 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b360bf-59ed-4909-8f84-e3376cc2c675 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:47.252290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:47.252963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:47.253247 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:47.253525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] PATH_INFO: `/v3/projects/3223775929a642b18e5fabff522e4bc6` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.255305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3223775929a642b18e5fabff522e4bc6)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:47.257860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:47.264701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1036/2048] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:47.269458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49522cea-c12b-472c-989d-70b048e69daa None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:47.301697 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49522cea-c12b-472c-989d-70b048e69daa None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:47.311407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304422c-b56c-46cc-a745-84c07012c7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:47.312610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304422c-b56c-46cc-a745-84c07012c7df admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:47.313117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304422c-b56c-46cc-a745-84c07012c7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:47.313670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304422c-b56c-46cc-a745-84c07012c7df admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.314940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304422c-b56c-46cc-a745-84c07012c7df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:47.318756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304422c-b56c-46cc-a745-84c07012c7df admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:47.322984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3223775929a642b18e5fabff522e4bc6', 'request_id': 'req-b8a69eff-d46d-496c-8401-e9201be71ce9'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:47.332469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3223775929a642b18e5fabff522e4bc6', 'request_id': 'req-b8a69eff-d46d-496c-8401-e9201be71ce9'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:47.334617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:47.340978 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:47.344127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3223775929a642b18e5fabff522e4bc6', 'request_id': 'req-b8a69eff-d46d-496c-8401-e9201be71ce9'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:47.363417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] The token cache is being invalidate because project 3223775929a642b18e5fabff522e4bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:47.364067 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:47.364574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b8a69eff-d46d-496c-8401-e9201be71ce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:47.366590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1013/2049] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:47 2022] DELETE /identity/v3/projects/3223775929a642b18e5fabff522e4bc6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:47.376380 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cbe20-f7e2-4282-b47c-2c32dbab0260 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:47.376851 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cbe20-f7e2-4282-b47c-2c32dbab0260 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:47.377269 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cbe20-f7e2-4282-b47c-2c32dbab0260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.382101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1037/2050] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:47 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:47.389486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304422c-b56c-46cc-a745-84c07012c7df admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:47.392385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c304422c-b56c-46cc-a745-84c07012c7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:47.404458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-209cbe20-f7e2-4282-b47c-2c32dbab0260 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:47.405851 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-209cbe20-f7e2-4282-b47c-2c32dbab0260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:47.513492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304422c-b56c-46cc-a745-84c07012c7df admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:47.521009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558c33d-043c-4df3-86ad-0445d43ef135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:47.522077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558c33d-043c-4df3-86ad-0445d43ef135 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:47.522558 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558c33d-043c-4df3-86ad-0445d43ef135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:47.523117 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558c33d-043c-4df3-86ad-0445d43ef135 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.523995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558c33d-043c-4df3-86ad-0445d43ef135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:47.526700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558c33d-043c-4df3-86ad-0445d43ef135 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:47.535937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1014/2051] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:47.539428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee981c-e120-41ff-819f-57fc3a85ea28 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:47.539726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee981c-e120-41ff-819f-57fc3a85ea28 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:47.540006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee981c-e120-41ff-819f-57fc3a85ea28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.568444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-75ee981c-e120-41ff-819f-57fc3a85ea28 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:47.570427 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-75ee981c-e120-41ff-819f-57fc3a85ea28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:47.571218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1038/2052] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:47 2022] POST /identity/v3/users => generated 295 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:47.579280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558c33d-043c-4df3-86ad-0445d43ef135 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:47.586485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1015/2053] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:47.592961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee981c-e120-41ff-819f-57fc3a85ea28 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:47.603352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558c33d-043c-4df3-86ad-0445d43ef135 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:47.610797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee981c-e120-41ff-819f-57fc3a85ea28 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:47.620241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30295989-3699-44f0-990a-15d65ba58b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:47.621327 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295989-3699-44f0-990a-15d65ba58b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:47.621838 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295989-3699-44f0-990a-15d65ba58b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:47.622336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295989-3699-44f0-990a-15d65ba58b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.622669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13a327-b1b2-4a4e-b32e-9c70bfc69de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:47.623601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295989-3699-44f0-990a-15d65ba58b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:47.623847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a327-b1b2-4a4e-b32e-9c70bfc69de5 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:47.624156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295989-3699-44f0-990a-15d65ba58b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:47.624532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a327-b1b2-4a4e-b32e-9c70bfc69de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:47.625017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a327-b1b2-4a4e-b32e-9c70bfc69de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.625897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13a327-b1b2-4a4e-b32e-9c70bfc69de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:47.628079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295989-3699-44f0-990a-15d65ba58b3a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:47.629825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13a327-b1b2-4a4e-b32e-9c70bfc69de5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:47.657790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1039/2054] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:47.662383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13a327-b1b2-4a4e-b32e-9c70bfc69de5 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:47.667292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1016/2055] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:08:47.672325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30295989-3699-44f0-990a-15d65ba58b3a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:47.674112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13a327-b1b2-4a4e-b32e-9c70bfc69de5 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:47.679901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd0f7c-59b1-4d90-aa23-27d618bc34ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:47.680740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd0f7c-59b1-4d90-aa23-27d618bc34ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:47.681171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd0f7c-59b1-4d90-aa23-27d618bc34ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:47.682772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd0f7c-59b1-4d90-aa23-27d618bc34ac admin admin] PATH_INFO: `/v3/projects/a5a19abb40484fda8856cfc9e12eee17/users/db619f8f7a184b678bbd37d6d4369df2/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.705276 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30295989-3699-44f0-990a-15d65ba58b3a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:47.715047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d63cef-629b-453b-bd71-7bcd0b3b4d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:47.715986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd0f7c-59b1-4d90-aa23-27d618bc34ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a19abb40484fda8856cfc9e12eee17, user_id=db619f8f7a184b678bbd37d6d4369df2, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:47.716139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d63cef-629b-453b-bd71-7bcd0b3b4d2b admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:47.716572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d63cef-629b-453b-bd71-7bcd0b3b4d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:47.716868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d63cef-629b-453b-bd71-7bcd0b3b4d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.717927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d63cef-629b-453b-bd71-7bcd0b3b4d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:47.720221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd0f7c-59b1-4d90-aa23-27d618bc34ac admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:47.721974 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d63cef-629b-453b-bd71-7bcd0b3b4d2b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:47.742994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1040/2056] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:47 2022] PUT /identity/v3/projects/a5a19abb40484fda8856cfc9e12eee17/users/db619f8f7a184b678bbd37d6d4369df2/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:47.749230 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd0f7c-59b1-4d90-aa23-27d618bc34ac admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:47.802406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1017/2057] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:47 2022] POST /identity/v3/projects => generated 397 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:47.804205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d63cef-629b-453b-bd71-7bcd0b3b4d2b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:47.867165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd0f7c-59b1-4d90-aa23-27d618bc34ac admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:47.946665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d63cef-629b-453b-bd71-7bcd0b3b4d2b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:47.964753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f79cbd-e0bd-405d-8d4d-c60697e15ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:47.965585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f79cbd-e0bd-405d-8d4d-c60697e15ffb admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:47.966541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f79cbd-e0bd-405d-8d4d-c60697e15ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:47.966826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f79cbd-e0bd-405d-8d4d-c60697e15ffb admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:47.967552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f79cbd-e0bd-405d-8d4d-c60697e15ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:47.976064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f79cbd-e0bd-405d-8d4d-c60697e15ffb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:47.989966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:48.016420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1018/2058] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:47 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:48.018927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f79cbd-e0bd-405d-8d4d-c60697e15ffb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:48.021180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0982db7-17c4-41e6-b68e-f471c71e6079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:48.032542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0982db7-17c4-41e6-b68e-f471c71e6079 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:48.033216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0982db7-17c4-41e6-b68e-f471c71e6079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:48.034570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0982db7-17c4-41e6-b68e-f471c71e6079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:48.035845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0982db7-17c4-41e6-b68e-f471c71e6079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:48.040311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0982db7-17c4-41e6-b68e-f471c71e6079 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:48.040490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f79cbd-e0bd-405d-8d4d-c60697e15ffb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:48.047044 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086016e-e9a0-45bd-9061-bdb9d7ac7fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:48.048321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086016e-e9a0-45bd-9061-bdb9d7ac7fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:48.048850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086016e-e9a0-45bd-9061-bdb9d7ac7fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:48.049517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086016e-e9a0-45bd-9061-bdb9d7ac7fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:48.051454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086016e-e9a0-45bd-9061-bdb9d7ac7fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:48.053814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086016e-e9a0-45bd-9061-bdb9d7ac7fe5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:48.075135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1041/2059] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:48.077029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1019/2060] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:48.079551 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0982db7-17c4-41e6-b68e-f471c71e6079 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:48.081655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086016e-e9a0-45bd-9061-bdb9d7ac7fe5 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:48.100245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086016e-e9a0-45bd-9061-bdb9d7ac7fe5 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:48.102504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0982db7-17c4-41e6-b68e-f471c71e6079 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:48.108004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7f623-195a-414e-b79a-10779bb95719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:48.108817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7f623-195a-414e-b79a-10779bb95719 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:48.109847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7f623-195a-414e-b79a-10779bb95719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:48.110155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7f623-195a-414e-b79a-10779bb95719 admin admin] PATH_INFO: `/v3/projects/a5a19abb40484fda8856cfc9e12eee17/users/db619f8f7a184b678bbd37d6d4369df2/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:48.113996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7f623-195a-414e-b79a-10779bb95719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a19abb40484fda8856cfc9e12eee17, user_id=db619f8f7a184b678bbd37d6d4369df2, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:48.118062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7f623-195a-414e-b79a-10779bb95719 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:48.129651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece41db-2b37-4d87-8e83-5abb30331971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:48.130872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece41db-2b37-4d87-8e83-5abb30331971 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:48.131701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece41db-2b37-4d87-8e83-5abb30331971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:48.132284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece41db-2b37-4d87-8e83-5abb30331971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:48.133908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece41db-2b37-4d87-8e83-5abb30331971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:48.139044 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1020/2061] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:48 2022] PUT /identity/v3/projects/a5a19abb40484fda8856cfc9e12eee17/users/db619f8f7a184b678bbd37d6d4369df2/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:48.139999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece41db-2b37-4d87-8e83-5abb30331971 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:48.153396 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b36cb-317a-48ce-bd8c-262a4c34e34f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:48.154750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b36cb-317a-48ce-bd8c-262a4c34e34f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:48.155261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b36cb-317a-48ce-bd8c-262a4c34e34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:48.178087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1042/2062] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:48.184424 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece41db-2b37-4d87-8e83-5abb30331971 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:48.198423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece41db-2b37-4d87-8e83-5abb30331971 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:48.200950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:48.203892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-812b36cb-317a-48ce-bd8c-262a4c34e34f None None] MFA Rules not processed for user `db619f8f7a184b678bbd37d6d4369df2`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:48.207177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-812b36cb-317a-48ce-bd8c-262a4c34e34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:48.292719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d512f6-e5b3-43e9-994c-8c94365427f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:48.293411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d512f6-e5b3-43e9-994c-8c94365427f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:48.293696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d512f6-e5b3-43e9-994c-8c94365427f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:48.294133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d512f6-e5b3-43e9-994c-8c94365427f2 admin admin] PATH_INFO: `/v3/projects/5c2fcb2e3b0442a18fd8d16a959c085a/users/ac77b88e0c6e40b39d0d9c3dbc3dddfa/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:48.337388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1021/2063] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:48.340210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d512f6-e5b3-43e9-994c-8c94365427f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c2fcb2e3b0442a18fd8d16a959c085a, user_id=ac77b88e0c6e40b39d0d9c3dbc3dddfa, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:48.345467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d512f6-e5b3-43e9-994c-8c94365427f2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:48.358182 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b36cb-317a-48ce-bd8c-262a4c34e34f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:48.358777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-812b36cb-317a-48ce-bd8c-262a4c34e34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:48.371503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1043/2064] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:48 2022] PUT /identity/v3/projects/5c2fcb2e3b0442a18fd8d16a959c085a/users/ac77b88e0c6e40b39d0d9c3dbc3dddfa/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:48.386899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf297d09-8d09-48aa-8c9d-95513969253e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:48.387268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf297d09-8d09-48aa-8c9d-95513969253e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:48.387582 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf297d09-8d09-48aa-8c9d-95513969253e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:48.431175 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bf297d09-8d09-48aa-8c9d-95513969253e None None] MFA Rules not processed for user `ac77b88e0c6e40b39d0d9c3dbc3dddfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:48.433750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bf297d09-8d09-48aa-8c9d-95513969253e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:48.437286 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b36cb-317a-48ce-bd8c-262a4c34e34f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:48.525843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cc4db-04aa-4ee7-be0f-444f757aef25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:48.526653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cc4db-04aa-4ee7-be0f-444f757aef25 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:48.526940 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cc4db-04aa-4ee7-be0f-444f757aef25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:48.527255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cc4db-04aa-4ee7-be0f-444f757aef25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:48.529473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-718cc4db-04aa-4ee7-be0f-444f757aef25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:48.554258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1044/2065] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:48.555198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf297d09-8d09-48aa-8c9d-95513969253e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:48.556765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bf297d09-8d09-48aa-8c9d-95513969253e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:48.615734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cc4db-04aa-4ee7-be0f-444f757aef25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:48.617277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cc4db-04aa-4ee7-be0f-444f757aef25 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:48.629698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1022/2066] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:08:48.630781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cc4db-04aa-4ee7-be0f-444f757aef25 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:48.632670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-718cc4db-04aa-4ee7-be0f-444f757aef25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:48.645030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:48.646124 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cc4db-04aa-4ee7-be0f-444f757aef25 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:48.651553 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d41e9-d9c7-406d-a038-32f89628fb56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:48.652435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d41e9-d9c7-406d-a038-32f89628fb56 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:48.652730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d41e9-d9c7-406d-a038-32f89628fb56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:48.652990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d41e9-d9c7-406d-a038-32f89628fb56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:48.660229 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:48.663333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d41e9-d9c7-406d-a038-32f89628fb56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:48.665292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d41e9-d9c7-406d-a038-32f89628fb56 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:48.674624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf297d09-8d09-48aa-8c9d-95513969253e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:48.676625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1023/2067] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:48 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:48.682988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c274a52-cd7e-4254-96dc-0265984b76c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:48.683656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c274a52-cd7e-4254-96dc-0265984b76c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:48.683937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c274a52-cd7e-4254-96dc-0265984b76c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:48.684212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c274a52-cd7e-4254-96dc-0265984b76c1 admin admin] PATH_INFO: `/v3/users/3725446c6d5040c7870b056d4bfbeca6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:48.685520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c274a52-cd7e-4254-96dc-0265984b76c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3725446c6d5040c7870b056d4bfbeca6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:48.687373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c274a52-cd7e-4254-96dc-0265984b76c1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:48.748964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4c274a52-cd7e-4254-96dc-0265984b76c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3725446c6d5040c7870b056d4bfbeca6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:48.769842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4c274a52-cd7e-4254-96dc-0265984b76c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3725446c6d5040c7870b056d4bfbeca6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:48.780893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4c274a52-cd7e-4254-96dc-0265984b76c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3725446c6d5040c7870b056d4bfbeca6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:48.791069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4c274a52-cd7e-4254-96dc-0265984b76c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3725446c6d5040c7870b056d4bfbeca6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:48.792223 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4c274a52-cd7e-4254-96dc-0265984b76c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3725446c6d5040c7870b056d4bfbeca6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:48.802150 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1045/2068] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:48 2022] DELETE /identity/v3/users/3725446c6d5040c7870b056d4bfbeca6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:49.068064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d41e9-d9c7-406d-a038-32f89628fb56 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:49.092313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d41e9-d9c7-406d-a038-32f89628fb56 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:49.198008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:49.199054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:49.199054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:49.199346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] PATH_INFO: `/v3/projects/70b4f8716fa84e439293149dd10cac82` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:49.200543 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70b4f8716fa84e439293149dd10cac82)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:49.202523 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:49.262184 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70b4f8716fa84e439293149dd10cac82', 'request_id': 'req-f3e0b678-7b6f-4535-945d-881e88cad4d4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:49.269211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70b4f8716fa84e439293149dd10cac82', 'request_id': 'req-f3e0b678-7b6f-4535-945d-881e88cad4d4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:49.278984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70b4f8716fa84e439293149dd10cac82', 'request_id': 'req-f3e0b678-7b6f-4535-945d-881e88cad4d4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:49.295706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] The token cache is being invalidate because project 70b4f8716fa84e439293149dd10cac82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:49.296004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:49.296682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3e0b678-7b6f-4535-945d-881e88cad4d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:49.298253 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1024/2069] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:49 2022] DELETE /identity/v3/projects/70b4f8716fa84e439293149dd10cac82 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:49.304670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d14a3-76ee-4226-b760-3707abc3c509 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:49.305566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d14a3-76ee-4226-b760-3707abc3c509 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:49.305770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d14a3-76ee-4226-b760-3707abc3c509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:49.340077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-fc9d14a3-76ee-4226-b760-3707abc3c509 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:49.341731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fc9d14a3-76ee-4226-b760-3707abc3c509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:49.475269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1046/2070] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:49.479058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6c17-ddd3-44c9-93de-ad9eb75b0663 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:49.480262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6c17-ddd3-44c9-93de-ad9eb75b0663 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:49.482651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6c17-ddd3-44c9-93de-ad9eb75b0663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:49.527230 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a8de6c17-ddd3-44c9-93de-ad9eb75b0663 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:49.529246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a8de6c17-ddd3-44c9-93de-ad9eb75b0663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:49.542921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1025/2071] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:49.548676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d14a3-76ee-4226-b760-3707abc3c509 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:49.561185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d14a3-76ee-4226-b760-3707abc3c509 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:49.567569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fa58f-912e-4e35-87a7-2fbd227df474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:49.568364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fa58f-912e-4e35-87a7-2fbd227df474 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:49.568700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fa58f-912e-4e35-87a7-2fbd227df474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:49.569121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fa58f-912e-4e35-87a7-2fbd227df474 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:49.569851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fa58f-912e-4e35-87a7-2fbd227df474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:49.571043 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fa58f-912e-4e35-87a7-2fbd227df474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:49.572264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fa58f-912e-4e35-87a7-2fbd227df474 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:49.599338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1047/2072] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:08:49.604912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6c17-ddd3-44c9-93de-ad9eb75b0663 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:49.615065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6c17-ddd3-44c9-93de-ad9eb75b0663 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:49.619981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af051b1-3eb7-41b9-a8ed-465168d4e562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:49.620594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af051b1-3eb7-41b9-a8ed-465168d4e562 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:49.620867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af051b1-3eb7-41b9-a8ed-465168d4e562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:49.621137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af051b1-3eb7-41b9-a8ed-465168d4e562 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:49.621793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af051b1-3eb7-41b9-a8ed-465168d4e562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:49.623741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af051b1-3eb7-41b9-a8ed-465168d4e562 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:49.661977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1026/2073] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:49 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:49.662419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fa58f-912e-4e35-87a7-2fbd227df474 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:49.681297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fa58f-912e-4e35-87a7-2fbd227df474 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:49.754172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:49.761457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba760a7-86d4-4a83-935f-c8b043de52d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:49.762693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba760a7-86d4-4a83-935f-c8b043de52d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:49.763672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba760a7-86d4-4a83-935f-c8b043de52d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:49.763967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba760a7-86d4-4a83-935f-c8b043de52d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:49.764817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba760a7-86d4-4a83-935f-c8b043de52d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:49.767352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba760a7-86d4-4a83-935f-c8b043de52d2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:49.806711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1048/2074] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:49 2022] POST /identity/v3/users => generated 302 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:49.813095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af051b1-3eb7-41b9-a8ed-465168d4e562 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:49.838921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af051b1-3eb7-41b9-a8ed-465168d4e562 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:49.846423 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cf38b-3c35-4cc7-837b-b9eb94f272a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:49.847107 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cf38b-3c35-4cc7-837b-b9eb94f272a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:49.847388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cf38b-3c35-4cc7-837b-b9eb94f272a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:49.847666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cf38b-3c35-4cc7-837b-b9eb94f272a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:49.848344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cf38b-3c35-4cc7-837b-b9eb94f272a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:49.849999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cf38b-3c35-4cc7-837b-b9eb94f272a8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:49.862370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1027/2075] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:49.867528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba760a7-86d4-4a83-935f-c8b043de52d2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:49.886277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba760a7-86d4-4a83-935f-c8b043de52d2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:49.894728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0794c1-4903-4d79-af82-69c7176c16d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:49.895401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0794c1-4903-4d79-af82-69c7176c16d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:49.895680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0794c1-4903-4d79-af82-69c7176c16d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:49.895974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0794c1-4903-4d79-af82-69c7176c16d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:49.896627 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0794c1-4903-4d79-af82-69c7176c16d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:49.898269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0794c1-4903-4d79-af82-69c7176c16d3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:49.911853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1049/2076] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:49.917620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cf38b-3c35-4cc7-837b-b9eb94f272a8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:49.928655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cf38b-3c35-4cc7-837b-b9eb94f272a8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:49.934513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee403522-c915-4676-b4c1-09edcac97011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:49.935230 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee403522-c915-4676-b4c1-09edcac97011 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:49.935505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee403522-c915-4676-b4c1-09edcac97011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:49.935773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee403522-c915-4676-b4c1-09edcac97011 admin admin] PATH_INFO: `/v3/projects/e3ec7d9fe5e34107b0d6ab0d4dc84022/users/9827756166b749e2be55a2b36e35ab89/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:49.954174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee403522-c915-4676-b4c1-09edcac97011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3ec7d9fe5e34107b0d6ab0d4dc84022, user_id=9827756166b749e2be55a2b36e35ab89, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:49.956910 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee403522-c915-4676-b4c1-09edcac97011 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:49.983255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1028/2077] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:49 2022] PUT /identity/v3/projects/e3ec7d9fe5e34107b0d6ab0d4dc84022/users/9827756166b749e2be55a2b36e35ab89/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:49.996634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c89bf5-a6bc-4ea3-a36b-24a568f7d0ef None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:49.997458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c89bf5-a6bc-4ea3-a36b-24a568f7d0ef None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:49.998199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c89bf5-a6bc-4ea3-a36b-24a568f7d0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:50.035420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-96c89bf5-a6bc-4ea3-a36b-24a568f7d0ef None None] MFA Rules not processed for user `9827756166b749e2be55a2b36e35ab89`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:50.037227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-96c89bf5-a6bc-4ea3-a36b-24a568f7d0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:50.135671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1050/2078] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:49 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:50.141342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee403522-c915-4676-b4c1-09edcac97011 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:50.182594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:50.242416 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee403522-c915-4676-b4c1-09edcac97011 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:50.251735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8454447-e1a6-4a39-a93f-efacb3637f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:50.252827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454447-e1a6-4a39-a93f-efacb3637f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:50.253351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454447-e1a6-4a39-a93f-efacb3637f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:50.253858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454447-e1a6-4a39-a93f-efacb3637f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:50.255140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8454447-e1a6-4a39-a93f-efacb3637f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:50.259021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8454447-e1a6-4a39-a93f-efacb3637f1a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:50.304854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1029/2079] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:50 2022] POST /identity/v3/projects => generated 405 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:50.309546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c89bf5-a6bc-4ea3-a36b-24a568f7d0ef None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:50.328680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c89bf5-a6bc-4ea3-a36b-24a568f7d0ef None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:50.407726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437745e-8c48-4d5a-90f4-06a791321c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:50.407726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437745e-8c48-4d5a-90f4-06a791321c0a admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:50.407726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437745e-8c48-4d5a-90f4-06a791321c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:50.407726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437745e-8c48-4d5a-90f4-06a791321c0a admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:50.408075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e437745e-8c48-4d5a-90f4-06a791321c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:50.409749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e437745e-8c48-4d5a-90f4-06a791321c0a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:50.436366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1051/2080] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:50 2022] POST /identity/v3/users => generated 309 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:50.440449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8454447-e1a6-4a39-a93f-efacb3637f1a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:50.454717 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8454447-e1a6-4a39-a93f-efacb3637f1a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:50.460593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437745e-8c48-4d5a-90f4-06a791321c0a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:50.461885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436e76a-ac7a-4db0-96c4-8427b77d8ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:50.462781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e76a-ac7a-4db0-96c4-8427b77d8ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:50.462924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e437745e-8c48-4d5a-90f4-06a791321c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:50.463143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e76a-ac7a-4db0-96c4-8427b77d8ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:50.463587 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e76a-ac7a-4db0-96c4-8427b77d8ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:50.464364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436e76a-ac7a-4db0-96c4-8427b77d8ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:50.466891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436e76a-ac7a-4db0-96c4-8427b77d8ae7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:50.486666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1030/2081] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:50.491030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436e76a-ac7a-4db0-96c4-8427b77d8ae7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:50.502453 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436e76a-ac7a-4db0-96c4-8427b77d8ae7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:50.508067 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442cd71-a083-4ecb-b9a6-3bb806d09c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:50.508955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442cd71-a083-4ecb-b9a6-3bb806d09c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:50.509374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442cd71-a083-4ecb-b9a6-3bb806d09c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:50.509723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442cd71-a083-4ecb-b9a6-3bb806d09c1e admin admin] PATH_INFO: `/v3/projects/15dcf73266874e60b8179a414fe4f98b/users/4abc7aa7ad15408c8127c8adef326581/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:50.527707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442cd71-a083-4ecb-b9a6-3bb806d09c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=15dcf73266874e60b8179a414fe4f98b, user_id=4abc7aa7ad15408c8127c8adef326581, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:50.531276 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442cd71-a083-4ecb-b9a6-3bb806d09c1e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:50.539174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437745e-8c48-4d5a-90f4-06a791321c0a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:50.544844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1d054-9916-41be-89e9-a472beff52b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:50.545666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1d054-9916-41be-89e9-a472beff52b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:50.545950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1d054-9916-41be-89e9-a472beff52b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:50.546224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1d054-9916-41be-89e9-a472beff52b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:50.548193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-44e1d054-9916-41be-89e9-a472beff52b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:50.553824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1031/2082] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:50 2022] PUT /identity/v3/projects/15dcf73266874e60b8179a414fe4f98b/users/4abc7aa7ad15408c8127c8adef326581/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:50.558444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442cd71-a083-4ecb-b9a6-3bb806d09c1e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:50.562993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:50.631915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442cd71-a083-4ecb-b9a6-3bb806d09c1e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:50.637752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1d054-9916-41be-89e9-a472beff52b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:50.639248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1d054-9916-41be-89e9-a472beff52b4 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:50.641820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c639b85-fe40-43d7-a4ce-1ce4ed33e100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:50.642384 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c639b85-fe40-43d7-a4ce-1ce4ed33e100 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:50.642663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c639b85-fe40-43d7-a4ce-1ce4ed33e100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:50.643204 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c639b85-fe40-43d7-a4ce-1ce4ed33e100 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:50.643939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c639b85-fe40-43d7-a4ce-1ce4ed33e100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:50.645807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c639b85-fe40-43d7-a4ce-1ce4ed33e100 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:50.654939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1052/2083] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:08:50.663020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1032/2084] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:08:50.666306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1d054-9916-41be-89e9-a472beff52b4 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:50.687464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1d054-9916-41be-89e9-a472beff52b4 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:50.694903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279267bb-a5a3-4655-9f2f-fdc57206d97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:50.695976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279267bb-a5a3-4655-9f2f-fdc57206d97d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:50.696455 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279267bb-a5a3-4655-9f2f-fdc57206d97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:50.696871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279267bb-a5a3-4655-9f2f-fdc57206d97d admin admin] PATH_INFO: `/v3/projects/15dcf73266874e60b8179a414fe4f98b/users/4abc7aa7ad15408c8127c8adef326581/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:50.701217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279267bb-a5a3-4655-9f2f-fdc57206d97d admin admin] RBAC: Authorizing `identity:create_grant(project_id=15dcf73266874e60b8179a414fe4f98b, user_id=4abc7aa7ad15408c8127c8adef326581, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:50.704576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279267bb-a5a3-4655-9f2f-fdc57206d97d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:50.717105 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1053/2085] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:50 2022] PUT /identity/v3/projects/15dcf73266874e60b8179a414fe4f98b/users/4abc7aa7ad15408c8127c8adef326581/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:50.729244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1b8ff-1855-46c0-858a-921a429da436 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:50.729534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1b8ff-1855-46c0-858a-921a429da436 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:50.729808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1b8ff-1855-46c0-858a-921a429da436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:50.764670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-15c1b8ff-1855-46c0-858a-921a429da436 None None] MFA Rules not processed for user `4abc7aa7ad15408c8127c8adef326581`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:50.766055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-15c1b8ff-1855-46c0-858a-921a429da436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:50.861344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:50.881627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1033/2086] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:50 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:50.894780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279267bb-a5a3-4655-9f2f-fdc57206d97d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:50.896890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-279267bb-a5a3-4655-9f2f-fdc57206d97d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:50.962391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279267bb-a5a3-4655-9f2f-fdc57206d97d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:50.966682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97457aaf-0590-432a-879b-5fd4afb2edd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:50.967325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97457aaf-0590-432a-879b-5fd4afb2edd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:50.967766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97457aaf-0590-432a-879b-5fd4afb2edd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:50.967909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97457aaf-0590-432a-879b-5fd4afb2edd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:50.979618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97457aaf-0590-432a-879b-5fd4afb2edd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:50.981360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97457aaf-0590-432a-879b-5fd4afb2edd1 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:50.995579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1054/2087] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:50 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:51.200216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1b8ff-1855-46c0-858a-921a429da436 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:51.284914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1b8ff-1855-46c0-858a-921a429da436 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:51.291144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:51.291747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:51.292038 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:51.292316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] PATH_INFO: `/v3/users/9827756166b749e2be55a2b36e35ab89` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:51.293567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9827756166b749e2be55a2b36e35ab89)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:51.295941 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:51.363830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9827756166b749e2be55a2b36e35ab89'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:51.386242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9827756166b749e2be55a2b36e35ab89'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:51.395108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9827756166b749e2be55a2b36e35ab89'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:51.396103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9827756166b749e2be55a2b36e35ab89'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:51.408254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9827756166b749e2be55a2b36e35ab89'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:51.415959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1034/2088] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:51 2022] DELETE /identity/v3/users/9827756166b749e2be55a2b36e35ab89 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:51.420206 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:51.421064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52748c2-e3fb-4775-a4bb-c62689a4f09d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:51.421890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:51.422567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52748c2-e3fb-4775-a4bb-c62689a4f09d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:51.423072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52748c2-e3fb-4775-a4bb-c62689a4f09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:51.445373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a52748c2-e3fb-4775-a4bb-c62689a4f09d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:51.451522 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a52748c2-e3fb-4775-a4bb-c62689a4f09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:51.506596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587536f-7ce9-472c-8f54-035504630083 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:51.513234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bab71-e3cb-4a77-8b14-9de2dc8bef25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:51.514278 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bab71-e3cb-4a77-8b14-9de2dc8bef25 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:51.514768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bab71-e3cb-4a77-8b14-9de2dc8bef25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:51.515295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bab71-e3cb-4a77-8b14-9de2dc8bef25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:51.518632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-554bab71-e3cb-4a77-8b14-9de2dc8bef25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:51.551372 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1055/2089] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:51.567919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:51.572744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52748c2-e3fb-4775-a4bb-c62689a4f09d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:51.575788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a52748c2-e3fb-4775-a4bb-c62689a4f09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:51.624498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bab71-e3cb-4a77-8b14-9de2dc8bef25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:51.625871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bab71-e3cb-4a77-8b14-9de2dc8bef25 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:51.636918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1035/2090] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:51.654586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:51.684634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52748c2-e3fb-4775-a4bb-c62689a4f09d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:51.690505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e54c9-d5ee-4d2f-888a-1c2126260e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:51.691512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e54c9-d5ee-4d2f-888a-1c2126260e01 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:51.691985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e54c9-d5ee-4d2f-888a-1c2126260e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:51.692456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e54c9-d5ee-4d2f-888a-1c2126260e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:51.710175 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e54c9-d5ee-4d2f-888a-1c2126260e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:51.712239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e54c9-d5ee-4d2f-888a-1c2126260e01 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:51.731633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1056/2091] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:52.169565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bab71-e3cb-4a77-8b14-9de2dc8bef25 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:52.170887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-554bab71-e3cb-4a77-8b14-9de2dc8bef25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:52.195658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bab71-e3cb-4a77-8b14-9de2dc8bef25 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:52.203681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:52.204859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:52.205347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:52.205791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] PATH_INFO: `/v3/projects/e3ec7d9fe5e34107b0d6ab0d4dc84022` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:52.207681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3ec7d9fe5e34107b0d6ab0d4dc84022)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:52.210569 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:52.281255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3ec7d9fe5e34107b0d6ab0d4dc84022', 'request_id': 'req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:52.289257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3ec7d9fe5e34107b0d6ab0d4dc84022', 'request_id': 'req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:52.299603 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3ec7d9fe5e34107b0d6ab0d4dc84022', 'request_id': 'req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:52.319343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] The token cache is being invalidate because project e3ec7d9fe5e34107b0d6ab0d4dc84022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:52.319777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:52.320745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:52.323870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1036/2092] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:52 2022] DELETE /identity/v3/projects/e3ec7d9fe5e34107b0d6ab0d4dc84022 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:52.329600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e54c9-d5ee-4d2f-888a-1c2126260e01 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:52.340390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e54c9-d5ee-4d2f-888a-1c2126260e01 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:52.424545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f819f9f-f14f-4fdb-8c85-9bb412214c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:52.425482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f819f9f-f14f-4fdb-8c85-9bb412214c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:52.425766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f819f9f-f14f-4fdb-8c85-9bb412214c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:52.426041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f819f9f-f14f-4fdb-8c85-9bb412214c91 admin admin] PATH_INFO: `/v3/users/4abc7aa7ad15408c8127c8adef326581` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:52.427305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f819f9f-f14f-4fdb-8c85-9bb412214c91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4abc7aa7ad15408c8127c8adef326581)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:52.429275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f819f9f-f14f-4fdb-8c85-9bb412214c91 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:52.499099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7f819f9f-f14f-4fdb-8c85-9bb412214c91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4abc7aa7ad15408c8127c8adef326581'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:52.521844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7f819f9f-f14f-4fdb-8c85-9bb412214c91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4abc7aa7ad15408c8127c8adef326581'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:52.532054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7f819f9f-f14f-4fdb-8c85-9bb412214c91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4abc7aa7ad15408c8127c8adef326581'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:52.541361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7f819f9f-f14f-4fdb-8c85-9bb412214c91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4abc7aa7ad15408c8127c8adef326581'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:52.542376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7f819f9f-f14f-4fdb-8c85-9bb412214c91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4abc7aa7ad15408c8127c8adef326581'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:52.549626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1057/2093] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:52 2022] DELETE /identity/v3/users/4abc7aa7ad15408c8127c8adef326581 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:52.865930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:52.869991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:52.904138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a0d4e-0afc-48e3-956b-f63518a2e9e8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:52.974429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:52.975008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:52.975290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:52.975559 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] PATH_INFO: `/v3/projects/15dcf73266874e60b8179a414fe4f98b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:52.976719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15dcf73266874e60b8179a414fe4f98b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:52.979387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:53.028167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:53.029696 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15dcf73266874e60b8179a414fe4f98b', 'request_id': 'req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:53.036774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15dcf73266874e60b8179a414fe4f98b', 'request_id': 'req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:53.045781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15dcf73266874e60b8179a414fe4f98b', 'request_id': 'req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:53.062383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] The token cache is being invalidate because project 15dcf73266874e60b8179a414fe4f98b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:08:53.062747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:53.063529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5e1c0247-94e0-4e62-b4fc-8bb8ab030713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:53.065366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1037/2094] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:52 2022] DELETE /identity/v3/projects/15dcf73266874e60b8179a414fe4f98b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:53.070228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1915462-7ed0-46c3-b341-c08dd4572a70 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:53.070515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1915462-7ed0-46c3-b341-c08dd4572a70 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:53.070887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1915462-7ed0-46c3-b341-c08dd4572a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:53.099208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f1915462-7ed0-46c3-b341-c08dd4572a70 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:53.099208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f1915462-7ed0-46c3-b341-c08dd4572a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:53.208039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1058/2095] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:53.217145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6554e-221b-4cb0-a249-1a8cbeebbcca None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:53.217614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6554e-221b-4cb0-a249-1a8cbeebbcca None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:53.218234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6554e-221b-4cb0-a249-1a8cbeebbcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:53.254903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-68d6554e-221b-4cb0-a249-1a8cbeebbcca None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:53.257501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-68d6554e-221b-4cb0-a249-1a8cbeebbcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:53.271195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1038/2096] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:53.278397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1915462-7ed0-46c3-b341-c08dd4572a70 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:53.287450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:53.298234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1915462-7ed0-46c3-b341-c08dd4572a70 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:53.308831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e26b9e-11d4-4977-bf91-526f61820ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:53.309990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e26b9e-11d4-4977-bf91-526f61820ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:53.310272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e26b9e-11d4-4977-bf91-526f61820ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:53.310556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e26b9e-11d4-4977-bf91-526f61820ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:53.311306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e26b9e-11d4-4977-bf91-526f61820ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:08:53.311644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e26b9e-11d4-4977-bf91-526f61820ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:53.313331 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e26b9e-11d4-4977-bf91-526f61820ee3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:53.326890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1059/2097] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:08:53.331707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6554e-221b-4cb0-a249-1a8cbeebbcca None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:53.342553 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6554e-221b-4cb0-a249-1a8cbeebbcca None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:53.347543 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60adc50c-7af8-4683-9e3b-b79ddf1eb5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:53.348188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adc50c-7af8-4683-9e3b-b79ddf1eb5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:53.348511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adc50c-7af8-4683-9e3b-b79ddf1eb5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:53.348780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adc50c-7af8-4683-9e3b-b79ddf1eb5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:53.349457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60adc50c-7af8-4683-9e3b-b79ddf1eb5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:53.351456 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60adc50c-7af8-4683-9e3b-b79ddf1eb5e0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:53.389432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1039/2098] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:53 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:53.394123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e26b9e-11d4-4977-bf91-526f61820ee3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:53.405788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e26b9e-11d4-4977-bf91-526f61820ee3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:53.468349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b443680-63cb-4990-93e4-9008ba4ee449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:53.469001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b443680-63cb-4990-93e4-9008ba4ee449 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:53.469222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b443680-63cb-4990-93e4-9008ba4ee449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:53.469501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b443680-63cb-4990-93e4-9008ba4ee449 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:53.470138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b443680-63cb-4990-93e4-9008ba4ee449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:53.472078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b443680-63cb-4990-93e4-9008ba4ee449 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:53.497090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1060/2099] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:53 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:53.501151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60adc50c-7af8-4683-9e3b-b79ddf1eb5e0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:53.512510 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60adc50c-7af8-4683-9e3b-b79ddf1eb5e0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:53.517544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ea6b5-b644-4b0e-b630-bd94e2d4dcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:53.518167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ea6b5-b644-4b0e-b630-bd94e2d4dcce admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:53.518445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ea6b5-b644-4b0e-b630-bd94e2d4dcce admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:53.519046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ea6b5-b644-4b0e-b630-bd94e2d4dcce admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:53.519706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ea6b5-b644-4b0e-b630-bd94e2d4dcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:53.521349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ea6b5-b644-4b0e-b630-bd94e2d4dcce admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:53.532606 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1040/2100] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:53.538357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b443680-63cb-4990-93e4-9008ba4ee449 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:53.549078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b443680-63cb-4990-93e4-9008ba4ee449 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:53.554274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13456e1f-2813-461c-998b-1787e038f489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:53.554887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13456e1f-2813-461c-998b-1787e038f489 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:53.555214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13456e1f-2813-461c-998b-1787e038f489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:53.555488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13456e1f-2813-461c-998b-1787e038f489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:53.556135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13456e1f-2813-461c-998b-1787e038f489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:53.558303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13456e1f-2813-461c-998b-1787e038f489 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:53.569682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1061/2101] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:08:53.575880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ea6b5-b644-4b0e-b630-bd94e2d4dcce admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:53.587166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ea6b5-b644-4b0e-b630-bd94e2d4dcce admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:53.592609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ecfd2-8ba5-4ff0-8da5-9b09ca0ba60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:53.593311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ecfd2-8ba5-4ff0-8da5-9b09ca0ba60b admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:53.593619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ecfd2-8ba5-4ff0-8da5-9b09ca0ba60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:53.593892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ecfd2-8ba5-4ff0-8da5-9b09ca0ba60b admin admin] PATH_INFO: `/v3/projects/2c39eb313b4f43d0b8e8b60730ed1dd9/users/2272ebf5d2f442f1abb081d9a4ea5b13/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:53.610767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ecfd2-8ba5-4ff0-8da5-9b09ca0ba60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c39eb313b4f43d0b8e8b60730ed1dd9, user_id=2272ebf5d2f442f1abb081d9a4ea5b13, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:53.613590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ecfd2-8ba5-4ff0-8da5-9b09ca0ba60b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:53.629084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1041/2102] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:53 2022] PUT /identity/v3/projects/2c39eb313b4f43d0b8e8b60730ed1dd9/users/2272ebf5d2f442f1abb081d9a4ea5b13/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:53.640998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e5319-68c8-43c2-b216-7700b75d0155 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:53.641419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e5319-68c8-43c2-b216-7700b75d0155 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:53.641811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e5319-68c8-43c2-b216-7700b75d0155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:53.671765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-de2e5319-68c8-43c2-b216-7700b75d0155 None None] MFA Rules not processed for user `2272ebf5d2f442f1abb081d9a4ea5b13`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:53.673046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-de2e5319-68c8-43c2-b216-7700b75d0155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:53.747827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1062/2103] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:53.753643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ecfd2-8ba5-4ff0-8da5-9b09ca0ba60b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:53.860515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ecfd2-8ba5-4ff0-8da5-9b09ca0ba60b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:53.866181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f18e65-e951-41ac-870d-86c7aadb4401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:53.867153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f18e65-e951-41ac-870d-86c7aadb4401 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:53.867153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f18e65-e951-41ac-870d-86c7aadb4401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:53.867800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f18e65-e951-41ac-870d-86c7aadb4401 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:53.868001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f18e65-e951-41ac-870d-86c7aadb4401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:53.870597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f18e65-e951-41ac-870d-86c7aadb4401 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:53.914760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1042/2104] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:08:53 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:08:53.919819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e5319-68c8-43c2-b216-7700b75d0155 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:53.937320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e5319-68c8-43c2-b216-7700b75d0155 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:54.029188 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:54.037215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fe289-d5b9-4dfe-bf9d-c0112bc8deef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:54.037883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fe289-d5b9-4dfe-bf9d-c0112bc8deef admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:54.038158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fe289-d5b9-4dfe-bf9d-c0112bc8deef admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:54.038451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fe289-d5b9-4dfe-bf9d-c0112bc8deef admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:54.039159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fe289-d5b9-4dfe-bf9d-c0112bc8deef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:54.041722 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fe289-d5b9-4dfe-bf9d-c0112bc8deef admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:54.065465 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1063/2105] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:08:53 2022] POST /identity/v3/users => generated 296 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:08:54.069675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f18e65-e951-41ac-870d-86c7aadb4401 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:54.081145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f18e65-e951-41ac-870d-86c7aadb4401 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:54.086460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0ec7-11ac-4d91-ba38-95b9ee6f9aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:54.087133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0ec7-11ac-4d91-ba38-95b9ee6f9aff admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:54.087406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0ec7-11ac-4d91-ba38-95b9ee6f9aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:54.088208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0ec7-11ac-4d91-ba38-95b9ee6f9aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:54.089008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e0ec7-11ac-4d91-ba38-95b9ee6f9aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:54.090688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e0ec7-11ac-4d91-ba38-95b9ee6f9aff admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:54.103233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1043/2106] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:54.108063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fe289-d5b9-4dfe-bf9d-c0112bc8deef admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:54.120157 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fe289-d5b9-4dfe-bf9d-c0112bc8deef admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:54.125489 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b4c90-a4d9-480e-a5e4-7ba0a7cd2c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:54.126295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4c90-a4d9-480e-a5e4-7ba0a7cd2c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:54.126683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4c90-a4d9-480e-a5e4-7ba0a7cd2c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:54.127106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4c90-a4d9-480e-a5e4-7ba0a7cd2c33 admin admin] PATH_INFO: `/v3/projects/5ee998789a464870ad18f41e14987bba/users/84e3ac09945f46c7bdd7e15f1d2094d8/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:54.144824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b4c90-a4d9-480e-a5e4-7ba0a7cd2c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee998789a464870ad18f41e14987bba, user_id=84e3ac09945f46c7bdd7e15f1d2094d8, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:54.147587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b4c90-a4d9-480e-a5e4-7ba0a7cd2c33 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:54.162356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1064/2107] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:54 2022] PUT /identity/v3/projects/5ee998789a464870ad18f41e14987bba/users/84e3ac09945f46c7bdd7e15f1d2094d8/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:54.167258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0ec7-11ac-4d91-ba38-95b9ee6f9aff admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:54.184958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0ec7-11ac-4d91-ba38-95b9ee6f9aff admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:54.259265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:54.276432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115eeb9a-1ad5-4f9c-a796-146d28d685df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:54.277134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115eeb9a-1ad5-4f9c-a796-146d28d685df admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:54.277475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115eeb9a-1ad5-4f9c-a796-146d28d685df admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:54.277787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115eeb9a-1ad5-4f9c-a796-146d28d685df admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:54.278590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115eeb9a-1ad5-4f9c-a796-146d28d685df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:54.280431 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115eeb9a-1ad5-4f9c-a796-146d28d685df admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:54.292272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1044/2108] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:08:54.297053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b4c90-a4d9-480e-a5e4-7ba0a7cd2c33 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:54.309171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b4c90-a4d9-480e-a5e4-7ba0a7cd2c33 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:54.315682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84444628-3d6f-45a5-b903-5e0ad80c660b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:54.316376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84444628-3d6f-45a5-b903-5e0ad80c660b admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:54.316640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84444628-3d6f-45a5-b903-5e0ad80c660b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:54.316916 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84444628-3d6f-45a5-b903-5e0ad80c660b admin admin] PATH_INFO: `/v3/projects/5ee998789a464870ad18f41e14987bba/users/84e3ac09945f46c7bdd7e15f1d2094d8/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:54.319128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84444628-3d6f-45a5-b903-5e0ad80c660b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee998789a464870ad18f41e14987bba, user_id=84e3ac09945f46c7bdd7e15f1d2094d8, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:54.321770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84444628-3d6f-45a5-b903-5e0ad80c660b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:54.333433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1065/2109] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:08:54 2022] PUT /identity/v3/projects/5ee998789a464870ad18f41e14987bba/users/84e3ac09945f46c7bdd7e15f1d2094d8/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:54.347761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2c1ef-28c9-412a-acbd-9c07e84596c5 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:54.347761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2c1ef-28c9-412a-acbd-9c07e84596c5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:54.347761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2c1ef-28c9-412a-acbd-9c07e84596c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:54.379872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a5d2c1ef-28c9-412a-acbd-9c07e84596c5 None None] MFA Rules not processed for user `84e3ac09945f46c7bdd7e15f1d2094d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:54.381302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a5d2c1ef-28c9-412a-acbd-9c07e84596c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:54.472628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1045/2110] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:54.486552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84444628-3d6f-45a5-b903-5e0ad80c660b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:54.488466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-84444628-3d6f-45a5-b903-5e0ad80c660b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:54.566697 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84444628-3d6f-45a5-b903-5e0ad80c660b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:54.570591 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c152bd7-f4a4-4c3a-bb5e-526f3f2b5f01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:54.571237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c152bd7-f4a4-4c3a-bb5e-526f3f2b5f01 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:54.571512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c152bd7-f4a4-4c3a-bb5e-526f3f2b5f01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:54.571780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c152bd7-f4a4-4c3a-bb5e-526f3f2b5f01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:54.581529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c152bd7-f4a4-4c3a-bb5e-526f3f2b5f01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:54.583012 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c152bd7-f4a4-4c3a-bb5e-526f3f2b5f01 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:54.594137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1066/2111] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:54.684756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2c1ef-28c9-412a-acbd-9c07e84596c5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:54.694824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2c1ef-28c9-412a-acbd-9c07e84596c5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:54.698374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f03f3-0d64-4251-aa0f-9efbaec54fc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:54.699005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f03f3-0d64-4251-aa0f-9efbaec54fc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:54.699284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f03f3-0d64-4251-aa0f-9efbaec54fc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:54.699552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f03f3-0d64-4251-aa0f-9efbaec54fc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:54.757547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f03f3-0d64-4251-aa0f-9efbaec54fc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:54.759036 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f03f3-0d64-4251-aa0f-9efbaec54fc2 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:54.768702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1046/2112] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:58.961500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c152bd7-f4a4-4c3a-bb5e-526f3f2b5f01 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:58.963651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6c152bd7-f4a4-4c3a-bb5e-526f3f2b5f01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:59.033618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c152bd7-f4a4-4c3a-bb5e-526f3f2b5f01 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:59.038354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:59.039012 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:59.039295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:59.039572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] PATH_INFO: `/v3/users/d2d572a9f71f488b9cb9d5254afbd512` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:59.040717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d572a9f71f488b9cb9d5254afbd512)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:59.042638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:59.092122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d572a9f71f488b9cb9d5254afbd512'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:59.098013 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:59.108505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d572a9f71f488b9cb9d5254afbd512'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:59.116729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d572a9f71f488b9cb9d5254afbd512'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:59.125676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d572a9f71f488b9cb9d5254afbd512'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:59.126646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d572a9f71f488b9cb9d5254afbd512'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:59.135613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1067/2113] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:58 2022] DELETE /identity/v3/users/d2d572a9f71f488b9cb9d5254afbd512 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:08:59.140359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c5f27-cb03-4f4a-ae07-e569161b2f34 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:59.140872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c5f27-cb03-4f4a-ae07-e569161b2f34 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:59.141031 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c5f27-cb03-4f4a-ae07-e569161b2f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:59.161213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-fd2c5f27-cb03-4f4a-ae07-e569161b2f34 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:59.162386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fd2c5f27-cb03-4f4a-ae07-e569161b2f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:59.243345 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1047/2114] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:08:59.257827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:59.259654 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:59.334634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7441bfd-99d5-4f9a-972e-d370a7655fbf admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:59.339719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b2588-ff58-40a1-870d-dcdec0972fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:59.340670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b2588-ff58-40a1-870d-dcdec0972fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:59.341158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b2588-ff58-40a1-870d-dcdec0972fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:59.341557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b2588-ff58-40a1-870d-dcdec0972fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:59.355129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b2588-ff58-40a1-870d-dcdec0972fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:59.357893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b2588-ff58-40a1-870d-dcdec0972fb1 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:59.377277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1068/2115] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:08:59.487449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c5f27-cb03-4f4a-ae07-e569161b2f34 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:59.489084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fd2c5f27-cb03-4f4a-ae07-e569161b2f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:59.506444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c5f27-cb03-4f4a-ae07-e569161b2f34 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:59.515293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:59.516544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:59.517135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:59.517728 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] PATH_INFO: `/v3/users/ac77b88e0c6e40b39d0d9c3dbc3dddfa` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:59.520073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac77b88e0c6e40b39d0d9c3dbc3dddfa)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:59.524336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:59.527499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:08:59.606144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac77b88e0c6e40b39d0d9c3dbc3dddfa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:59.626785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac77b88e0c6e40b39d0d9c3dbc3dddfa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:59.643492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac77b88e0c6e40b39d0d9c3dbc3dddfa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:59.645548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac77b88e0c6e40b39d0d9c3dbc3dddfa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:59.661495 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac77b88e0c6e40b39d0d9c3dbc3dddfa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:08:59.671204 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1048/2116] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:08:59 2022] DELETE /identity/v3/users/ac77b88e0c6e40b39d0d9c3dbc3dddfa => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:08:59.676602 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b900-b357-488f-ba00-fc29900a1eb8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:59.677050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b900-b357-488f-ba00-fc29900a1eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:59.677551 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b900-b357-488f-ba00-fc29900a1eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:59.702634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1a43b900-b357-488f-ba00-fc29900a1eb8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:08:59.703802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1a43b900-b357-488f-ba00-fc29900a1eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:59.794344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1069/2117] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:08:59.805734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:59.808623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:59.889983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43b900-b357-488f-ba00-fc29900a1eb8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:08:59.892631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1a43b900-b357-488f-ba00-fc29900a1eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:08:59.909794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e0208-3af2-4ecd-96ac-bece1e89dcc6 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:59.914149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cde22e-1210-488c-a913-91241e9437e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:59.914149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cde22e-1210-488c-a913-91241e9437e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:59.914402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cde22e-1210-488c-a913-91241e9437e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:59.914646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cde22e-1210-488c-a913-91241e9437e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:59.917646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43b900-b357-488f-ba00-fc29900a1eb8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:08:59.927260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cde22e-1210-488c-a913-91241e9437e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:59.927917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:08:59.928729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cde22e-1210-488c-a913-91241e9437e9 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:59.929184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:08:59.930417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:08:59.930800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] PATH_INFO: `/v3/projects/c59b0234992042d5acb002ebabc09da5` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:08:59.932529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c59b0234992042d5acb002ebabc09da5)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:08:59.935813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:08:59.940724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1049/2118] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:09:00.029658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c59b0234992042d5acb002ebabc09da5', 'request_id': 'req-2c87a274-1a05-4dd2-b611-4a22a24f5298'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.043434 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c59b0234992042d5acb002ebabc09da5', 'request_id': 'req-2c87a274-1a05-4dd2-b611-4a22a24f5298'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.057087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c59b0234992042d5acb002ebabc09da5', 'request_id': 'req-2c87a274-1a05-4dd2-b611-4a22a24f5298'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.078884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:00.085022 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] The token cache is being invalidate because project c59b0234992042d5acb002ebabc09da5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:09:00.085319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.086379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.089915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1070/2119] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:08:59 2022] DELETE /identity/v3/projects/c59b0234992042d5acb002ebabc09da5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:09:00.094439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cde22e-1210-488c-a913-91241e9437e9 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:00.110649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cde22e-1210-488c-a913-91241e9437e9 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:00.203165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2207adf-d963-4360-ae80-645997b6c75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:00.204304 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2207adf-d963-4360-ae80-645997b6c75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:00.204752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2207adf-d963-4360-ae80-645997b6c75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:00.205229 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2207adf-d963-4360-ae80-645997b6c75e admin admin] PATH_INFO: `/v3/users/39bb99aa285f4d9ab099f7c8f915449f` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:00.207406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2207adf-d963-4360-ae80-645997b6c75e admin admin] RBAC: Authorizing `identity:delete_user(user_id=39bb99aa285f4d9ab099f7c8f915449f)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:00.211118 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2207adf-d963-4360-ae80-645997b6c75e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:00.248357 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:00.284630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f2207adf-d963-4360-ae80-645997b6c75e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39bb99aa285f4d9ab099f7c8f915449f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.306605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f2207adf-d963-4360-ae80-645997b6c75e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39bb99aa285f4d9ab099f7c8f915449f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.320294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:00.322904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:00.324768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f2207adf-d963-4360-ae80-645997b6c75e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39bb99aa285f4d9ab099f7c8f915449f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.327111 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f2207adf-d963-4360-ae80-645997b6c75e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39bb99aa285f4d9ab099f7c8f915449f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.342285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f2207adf-d963-4360-ae80-645997b6c75e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39bb99aa285f4d9ab099f7c8f915449f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.351610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1050/2120] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:09:00 2022] DELETE /identity/v3/users/39bb99aa285f4d9ab099f7c8f915449f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:00.353835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87a274-1a05-4dd2-b611-4a22a24f5298 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:00.461257 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:00.462421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:00.463737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:00.463926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] PATH_INFO: `/v3/projects/5c2fcb2e3b0442a18fd8d16a959c085a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:00.466664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c2fcb2e3b0442a18fd8d16a959c085a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:00.472926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:00.565050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c2fcb2e3b0442a18fd8d16a959c085a', 'request_id': 'req-e2b17c99-1f63-4c5e-933d-c96a2f074de3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.576140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c2fcb2e3b0442a18fd8d16a959c085a', 'request_id': 'req-e2b17c99-1f63-4c5e-933d-c96a2f074de3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.584644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c2fcb2e3b0442a18fd8d16a959c085a', 'request_id': 'req-e2b17c99-1f63-4c5e-933d-c96a2f074de3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.609532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] The token cache is being invalidate because project 5c2fcb2e3b0442a18fd8d16a959c085a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:09:00.609532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.615719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e2b17c99-1f63-4c5e-933d-c96a2f074de3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.621548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1071/2121] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:09:00 2022] DELETE /identity/v3/projects/5c2fcb2e3b0442a18fd8d16a959c085a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:09:00.628951 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a43118-201b-431a-bd81-e246a4cb4dc6 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:00.629400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a43118-201b-431a-bd81-e246a4cb4dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:00.629762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a43118-201b-431a-bd81-e246a4cb4dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:00.659734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d2a43118-201b-431a-bd81-e246a4cb4dc6 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:00.660994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d2a43118-201b-431a-bd81-e246a4cb4dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:00.746441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1051/2122] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:00.756619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd5fdf-3e78-4201-b47d-c3038a882430 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:00.757086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd5fdf-3e78-4201-b47d-c3038a882430 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:00.757606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd5fdf-3e78-4201-b47d-c3038a882430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:00.792828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-96cd5fdf-3e78-4201-b47d-c3038a882430 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:00.795516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-96cd5fdf-3e78-4201-b47d-c3038a882430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:00.808860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1072/2123] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:00.810383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a43118-201b-431a-bd81-e246a4cb4dc6 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:00.823106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a43118-201b-431a-bd81-e246a4cb4dc6 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:00.828976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb7609-f4a4-4f35-90c9-0591f040e186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:00.829591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb7609-f4a4-4f35-90c9-0591f040e186 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:00.829878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb7609-f4a4-4f35-90c9-0591f040e186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:00.830333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb7609-f4a4-4f35-90c9-0591f040e186 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:00.831116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb7609-f4a4-4f35-90c9-0591f040e186 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:09:00.831434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb7609-f4a4-4f35-90c9-0591f040e186 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:00.833123 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb7609-f4a4-4f35-90c9-0591f040e186 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:00.851140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1052/2124] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:09:00.855621 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd5fdf-3e78-4201-b47d-c3038a882430 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:00.866390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb7609-f4a4-4f35-90c9-0591f040e186 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:00.869073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-55cb7609-f4a4-4f35-90c9-0591f040e186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:00.869752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd5fdf-3e78-4201-b47d-c3038a882430 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:00.875476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b14738-58b3-45e3-8aa0-2e576a56808b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:00.876198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b14738-58b3-45e3-8aa0-2e576a56808b admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:00.876549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b14738-58b3-45e3-8aa0-2e576a56808b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:00.876933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b14738-58b3-45e3-8aa0-2e576a56808b admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:00.877693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b14738-58b3-45e3-8aa0-2e576a56808b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:00.880468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b14738-58b3-45e3-8aa0-2e576a56808b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:00.883074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb7609-f4a4-4f35-90c9-0591f040e186 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:00.890454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:00.891218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:00.891498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:00.891772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] PATH_INFO: `/v3/projects/1dd3e58a14ac4b0d847f8c21550fd5fa` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:00.893038 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd3e58a14ac4b0d847f8c21550fd5fa)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:00.895386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:00.933529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1073/2125] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:09:00 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:00.934974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b14738-58b3-45e3-8aa0-2e576a56808b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:00.968112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd3e58a14ac4b0d847f8c21550fd5fa', 'request_id': 'req-8d0f5142-d381-4bde-bb42-2aca517a6786'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.977768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd3e58a14ac4b0d847f8c21550fd5fa', 'request_id': 'req-8d0f5142-d381-4bde-bb42-2aca517a6786'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:00.987233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd3e58a14ac4b0d847f8c21550fd5fa', 'request_id': 'req-8d0f5142-d381-4bde-bb42-2aca517a6786'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.003694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] The token cache is being invalidate because project 1dd3e58a14ac4b0d847f8c21550fd5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:09:01.004067 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.010651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.010651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1053/2126] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:09:00 2022] DELETE /identity/v3/projects/1dd3e58a14ac4b0d847f8c21550fd5fa => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:09:01.012641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:01.014136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:01.028856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:01.035338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b14738-58b3-45e3-8aa0-2e576a56808b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:01.042328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:01.116548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f5142-d381-4bde-bb42-2aca517a6786 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:01.129550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:01.129885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:01.130127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:01.130411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] PATH_INFO: `/v3/users/db619f8f7a184b678bbd37d6d4369df2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:01.131653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] RBAC: Authorizing `identity:delete_user(user_id=db619f8f7a184b678bbd37d6d4369df2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:01.133444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b9aee-2499-443d-9f6d-087030addd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:01.133627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:01.134456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b9aee-2499-443d-9f6d-087030addd1e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:01.134994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b9aee-2499-443d-9f6d-087030addd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:01.135850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b9aee-2499-443d-9f6d-087030addd1e admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:01.137870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b9aee-2499-443d-9f6d-087030addd1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:01.146403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b9aee-2499-443d-9f6d-087030addd1e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:01.197690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1074/2127] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:09:00 2022] POST /identity/v3/users => generated 287 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:09:01.197690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b9aee-2499-443d-9f6d-087030addd1e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:01.204560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1d8b9aee-2499-443d-9f6d-087030addd1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:01.205590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db619f8f7a184b678bbd37d6d4369df2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.227059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db619f8f7a184b678bbd37d6d4369df2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.231828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b9aee-2499-443d-9f6d-087030addd1e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:01.237575 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db619f8f7a184b678bbd37d6d4369df2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.238548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db619f8f7a184b678bbd37d6d4369df2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.240618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa6e27-6118-4865-a85f-53d1fea249d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:01.241729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa6e27-6118-4865-a85f-53d1fea249d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:01.242283 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa6e27-6118-4865-a85f-53d1fea249d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:01.242845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa6e27-6118-4865-a85f-53d1fea249d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:01.244159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa6e27-6118-4865-a85f-53d1fea249d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:01.247496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa6e27-6118-4865-a85f-53d1fea249d3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:01.252628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db619f8f7a184b678bbd37d6d4369df2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.261747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1054/2128] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:09:01 2022] DELETE /identity/v3/users/db619f8f7a184b678bbd37d6d4369df2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:09:01.271037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1075/2129] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:01.276473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:01.278971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:01.364953 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a912-a02b-4cbc-b828-477bc07b950b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:01.370404 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e9b4e-a257-4dff-a338-83cc81e75939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:01.370891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e9b4e-a257-4dff-a338-83cc81e75939 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:01.371221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e9b4e-a257-4dff-a338-83cc81e75939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:01.371547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e9b4e-a257-4dff-a338-83cc81e75939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:01.372211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e9b4e-a257-4dff-a338-83cc81e75939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:01.373860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e9b4e-a257-4dff-a338-83cc81e75939 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:01.385964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1055/2130] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:01.391035 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa6e27-6118-4865-a85f-53d1fea249d3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:01.414105 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa6e27-6118-4865-a85f-53d1fea249d3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:01.423320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b713baf-a8eb-4739-b813-37f64ec56681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:01.424395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b713baf-a8eb-4739-b813-37f64ec56681 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:01.424849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b713baf-a8eb-4739-b813-37f64ec56681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:01.425533 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b713baf-a8eb-4739-b813-37f64ec56681 admin admin] PATH_INFO: `/v3/projects/1ed7377d32844bbf95f995831bc21d6b/users/0365ab1bcfd54946b0b92f8fe9697128/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:01.451083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b713baf-a8eb-4739-b813-37f64ec56681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ed7377d32844bbf95f995831bc21d6b, user_id=0365ab1bcfd54946b0b92f8fe9697128, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:01.456235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b713baf-a8eb-4739-b813-37f64ec56681 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:01.478713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1076/2131] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:01 2022] PUT /identity/v3/projects/1ed7377d32844bbf95f995831bc21d6b/users/0365ab1bcfd54946b0b92f8fe9697128/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:09:01.485829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae7bcb-9758-4952-a197-8c55cb726bed None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:01.486243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae7bcb-9758-4952-a197-8c55cb726bed None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:01.486599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae7bcb-9758-4952-a197-8c55cb726bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:01.516405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-47ae7bcb-9758-4952-a197-8c55cb726bed None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:01.518689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-47ae7bcb-9758-4952-a197-8c55cb726bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:01.553449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:01.614499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1056/2132] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:01.626704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b713baf-a8eb-4739-b813-37f64ec56681 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:01.629302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2b713baf-a8eb-4739-b813-37f64ec56681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:01.674505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae7bcb-9758-4952-a197-8c55cb726bed None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:01.676486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-47ae7bcb-9758-4952-a197-8c55cb726bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:01.690135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae7bcb-9758-4952-a197-8c55cb726bed None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:01.696958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:01.697699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:01.698250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:01.698751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] PATH_INFO: `/v3/projects/a5a19abb40484fda8856cfc9e12eee17` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:01.701720 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a19abb40484fda8856cfc9e12eee17)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:01.705754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:01.713112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b713baf-a8eb-4739-b813-37f64ec56681 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:01.719858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49ba2e-8a0b-4825-9838-cfd88991f15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:01.720797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49ba2e-8a0b-4825-9838-cfd88991f15e service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:01.721227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49ba2e-8a0b-4825-9838-cfd88991f15e service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:01.721640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49ba2e-8a0b-4825-9838-cfd88991f15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:01.738576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49ba2e-8a0b-4825-9838-cfd88991f15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:01.741485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49ba2e-8a0b-4825-9838-cfd88991f15e service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:01.762753 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1077/2133] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:01.795739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a19abb40484fda8856cfc9e12eee17', 'request_id': 'req-fef15bb5-8b81-415f-89d3-b370b2790e13'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.806567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a19abb40484fda8856cfc9e12eee17', 'request_id': 'req-fef15bb5-8b81-415f-89d3-b370b2790e13'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.818346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a19abb40484fda8856cfc9e12eee17', 'request_id': 'req-fef15bb5-8b81-415f-89d3-b370b2790e13'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.840362 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] The token cache is being invalidate because project a5a19abb40484fda8856cfc9e12eee17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:09:01.840827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.842396 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fef15bb5-8b81-415f-89d3-b370b2790e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:01.845084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1057/2134] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:09:01 2022] DELETE /identity/v3/projects/a5a19abb40484fda8856cfc9e12eee17 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:09:01.852761 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d54dbd-211b-406b-8b3b-975a6f73925e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:01.853883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d54dbd-211b-406b-8b3b-975a6f73925e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:01.854386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d54dbd-211b-406b-8b3b-975a6f73925e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:01.885985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-75d54dbd-211b-406b-8b3b-975a6f73925e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:01.888435 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-75d54dbd-211b-406b-8b3b-975a6f73925e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:02.037432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1078/2135] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:02.049061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b5629-ba0c-477e-bfb8-06b31cef5c63 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:02.049767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b5629-ba0c-477e-bfb8-06b31cef5c63 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:02.051143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b5629-ba0c-477e-bfb8-06b31cef5c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:02.082087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-476b5629-ba0c-477e-bfb8-06b31cef5c63 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:02.083578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-476b5629-ba0c-477e-bfb8-06b31cef5c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:02.093559 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1058/2136] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:02.102246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d54dbd-211b-406b-8b3b-975a6f73925e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:02.115185 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:02.124684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d54dbd-211b-406b-8b3b-975a6f73925e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:02.136986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed5760-9548-4cfa-9b93-d035b3b2dc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:02.138322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed5760-9548-4cfa-9b93-d035b3b2dc07 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:02.138799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed5760-9548-4cfa-9b93-d035b3b2dc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:02.139378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed5760-9548-4cfa-9b93-d035b3b2dc07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:02.140537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed5760-9548-4cfa-9b93-d035b3b2dc07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:09:02.141572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed5760-9548-4cfa-9b93-d035b3b2dc07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:02.145473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed5760-9548-4cfa-9b93-d035b3b2dc07 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:02.186674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1079/2137] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:09:02.187750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b5629-ba0c-477e-bfb8-06b31cef5c63 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:02.206058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b5629-ba0c-477e-bfb8-06b31cef5c63 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:02.214877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2cf06-b291-4326-80c9-16152d86da89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:02.215820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2cf06-b291-4326-80c9-16152d86da89 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:02.216071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2cf06-b291-4326-80c9-16152d86da89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:02.216444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2cf06-b291-4326-80c9-16152d86da89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:02.217302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2cf06-b291-4326-80c9-16152d86da89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:02.229342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2cf06-b291-4326-80c9-16152d86da89 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:02.264006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1059/2138] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:09:02 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:09:02.273146 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed5760-9548-4cfa-9b93-d035b3b2dc07 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:02.296674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed5760-9548-4cfa-9b93-d035b3b2dc07 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:02.394034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f44991-0974-4a53-b020-7b25b0033025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:02.394737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f44991-0974-4a53-b020-7b25b0033025 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:02.395140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f44991-0974-4a53-b020-7b25b0033025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:02.395498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f44991-0974-4a53-b020-7b25b0033025 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:02.396373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f44991-0974-4a53-b020-7b25b0033025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:02.398851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f44991-0974-4a53-b020-7b25b0033025 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:02.432063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1080/2139] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:09:02 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:02.436734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2cf06-b291-4326-80c9-16152d86da89 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:02.447572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2cf06-b291-4326-80c9-16152d86da89 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:02.452571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6ab71-d61f-43c3-b0d5-9eb5dcd80d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:02.453204 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6ab71-d61f-43c3-b0d5-9eb5dcd80d40 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:02.453481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6ab71-d61f-43c3-b0d5-9eb5dcd80d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:02.453751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6ab71-d61f-43c3-b0d5-9eb5dcd80d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:02.454402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6ab71-d61f-43c3-b0d5-9eb5dcd80d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:02.456071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6ab71-d61f-43c3-b0d5-9eb5dcd80d40 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:02.469688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1060/2140] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:02.473798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f44991-0974-4a53-b020-7b25b0033025 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:02.494707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f44991-0974-4a53-b020-7b25b0033025 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:02.500024 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334fa3-17bf-4ac8-8cf4-4c0b3fc8fdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:02.500652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334fa3-17bf-4ac8-8cf4-4c0b3fc8fdb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:02.500952 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334fa3-17bf-4ac8-8cf4-4c0b3fc8fdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:02.501191 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334fa3-17bf-4ac8-8cf4-4c0b3fc8fdb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:02.501853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40334fa3-17bf-4ac8-8cf4-4c0b3fc8fdb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:02.503520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40334fa3-17bf-4ac8-8cf4-4c0b3fc8fdb0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:02.515483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1081/2141] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:02.520171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6ab71-d61f-43c3-b0d5-9eb5dcd80d40 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:02.534557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6ab71-d61f-43c3-b0d5-9eb5dcd80d40 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:02.540497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ef7c9-5442-4c35-a0a9-869436a546af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:02.541947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ef7c9-5442-4c35-a0a9-869436a546af admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:02.542323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ef7c9-5442-4c35-a0a9-869436a546af admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:02.542769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ef7c9-5442-4c35-a0a9-869436a546af admin admin] PATH_INFO: `/v3/projects/6c516bf5715a42cd860981f82fc58b15/users/2ad9fb3dd77d47c190e14cf504ea4d3d/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:02.563699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ef7c9-5442-4c35-a0a9-869436a546af admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c516bf5715a42cd860981f82fc58b15, user_id=2ad9fb3dd77d47c190e14cf504ea4d3d, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:02.567313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ef7c9-5442-4c35-a0a9-869436a546af admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:02.585311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1061/2142] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:02 2022] PUT /identity/v3/projects/6c516bf5715a42cd860981f82fc58b15/users/2ad9fb3dd77d47c190e14cf504ea4d3d/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:02.597390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0edc50-c6cc-4339-9243-8ab98e069c6f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:02.598001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0edc50-c6cc-4339-9243-8ab98e069c6f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:02.598618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0edc50-c6cc-4339-9243-8ab98e069c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:02.624079 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0c0edc50-c6cc-4339-9243-8ab98e069c6f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:02.625329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0c0edc50-c6cc-4339-9243-8ab98e069c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:02.699275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1082/2143] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:02.709890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ef7c9-5442-4c35-a0a9-869436a546af admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:02.713291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bb6ef7c9-5442-4c35-a0a9-869436a546af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:02.817793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:02.825024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ef7c9-5442-4c35-a0a9-869436a546af admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:02.833378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3b5a2-59f4-45f9-8090-16ec67151eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:02.837266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3b5a2-59f4-45f9-8090-16ec67151eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:02.837901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3b5a2-59f4-45f9-8090-16ec67151eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:02.838381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3b5a2-59f4-45f9-8090-16ec67151eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:02.862059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3b5a2-59f4-45f9-8090-16ec67151eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:02.864756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3b5a2-59f4-45f9-8090-16ec67151eb5 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:02.878392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1062/2144] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:09:04.976857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729285e-f68e-482b-9800-9420590e644b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:04.977347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729285e-f68e-482b-9800-9420590e644b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:04.977449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729285e-f68e-482b-9800-9420590e644b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:05.038039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6729285e-f68e-482b-9800-9420590e644b None None] MFA Rules not processed for user `2ad9fb3dd77d47c190e14cf504ea4d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:05.040802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6729285e-f68e-482b-9800-9420590e644b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:05.147589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:05.157539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1083/2145] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:05.172143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3b5a2-59f4-45f9-8090-16ec67151eb5 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:05.173652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-96b3b5a2-59f4-45f9-8090-16ec67151eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:05.282700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3b5a2-59f4-45f9-8090-16ec67151eb5 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:05.290044 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27248c-9494-44fb-935c-0038541805fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:05.291367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27248c-9494-44fb-935c-0038541805fe service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:05.291645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27248c-9494-44fb-935c-0038541805fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:05.291914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27248c-9494-44fb-935c-0038541805fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:05.320910 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27248c-9494-44fb-935c-0038541805fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:05.322933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27248c-9494-44fb-935c-0038541805fe service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:05.359188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1063/2146] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:06.996838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374733c6-7179-40c6-9ed6-4df506c9eb3d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:06.998054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374733c6-7179-40c6-9ed6-4df506c9eb3d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:06.998581 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374733c6-7179-40c6-9ed6-4df506c9eb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:07.047325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-374733c6-7179-40c6-9ed6-4df506c9eb3d None None] MFA Rules not processed for user `2ad9fb3dd77d47c190e14cf504ea4d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:07.048816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-374733c6-7179-40c6-9ed6-4df506c9eb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:07.073268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1084/2147] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:08.139245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27248c-9494-44fb-935c-0038541805fe service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:08.141567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2d27248c-9494-44fb-935c-0038541805fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:08.202188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27248c-9494-44fb-935c-0038541805fe service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:08.231322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:08.232019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:08.232307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:08.232585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] PATH_INFO: `/v3/users/2272ebf5d2f442f1abb081d9a4ea5b13` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:08.234593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2272ebf5d2f442f1abb081d9a4ea5b13)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:08.238280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:08.386302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2272ebf5d2f442f1abb081d9a4ea5b13'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:08.418061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2272ebf5d2f442f1abb081d9a4ea5b13'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:08.435393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2272ebf5d2f442f1abb081d9a4ea5b13'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:08.437303 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2272ebf5d2f442f1abb081d9a4ea5b13'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:08.455115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2272ebf5d2f442f1abb081d9a4ea5b13'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:08.474489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1064/2148] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:09:08 2022] DELETE /identity/v3/users/2272ebf5d2f442f1abb081d9a4ea5b13 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:08.485153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d721bd-0ceb-436d-bd9e-25284193e646 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:08.485727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d721bd-0ceb-436d-bd9e-25284193e646 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:08.485850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d721bd-0ceb-436d-bd9e-25284193e646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:08.517816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d0d721bd-0ceb-436d-bd9e-25284193e646 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:08.519714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d0d721bd-0ceb-436d-bd9e-25284193e646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:08.677881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1085/2149] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:08.696352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:08.701489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:08.898655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183f746-14ba-4894-986b-d496ff8f6fa8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:08.909276 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01383f7a-ef1d-4f4b-a56b-98a4769656df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:08.911213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01383f7a-ef1d-4f4b-a56b-98a4769656df service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:08.912223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01383f7a-ef1d-4f4b-a56b-98a4769656df service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:08.912897 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01383f7a-ef1d-4f4b-a56b-98a4769656df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:08.952263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01383f7a-ef1d-4f4b-a56b-98a4769656df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:08.955995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01383f7a-ef1d-4f4b-a56b-98a4769656df service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:08.991421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1065/2150] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:09.009254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f330ad2-41cd-4afe-98ee-046e222bea4e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:09.010070 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f330ad2-41cd-4afe-98ee-046e222bea4e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:09.010759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f330ad2-41cd-4afe-98ee-046e222bea4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:09.096236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5f330ad2-41cd-4afe-98ee-046e222bea4e None None] MFA Rules not processed for user `0365ab1bcfd54946b0b92f8fe9697128`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:09.100446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5f330ad2-41cd-4afe-98ee-046e222bea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:09.247101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1086/2151] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:09.254172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01383f7a-ef1d-4f4b-a56b-98a4769656df service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:09.255664 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-01383f7a-ef1d-4f4b-a56b-98a4769656df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:09.284885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01383f7a-ef1d-4f4b-a56b-98a4769656df service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:09.299420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c579e1-27f8-4317-a696-391e74dc6cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:09.302497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c579e1-27f8-4317-a696-391e74dc6cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:09.303254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c579e1-27f8-4317-a696-391e74dc6cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:09.303754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c579e1-27f8-4317-a696-391e74dc6cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:09.305548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c579e1-27f8-4317-a696-391e74dc6cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:09.311149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c579e1-27f8-4317-a696-391e74dc6cb7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:09.319689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:09.357782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1066/2152] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:09:09 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:09:09.362092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f330ad2-41cd-4afe-98ee-046e222bea4e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:09.375021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f330ad2-41cd-4afe-98ee-046e222bea4e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:09.536022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c579e1-27f8-4317-a696-391e74dc6cb7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:09.538823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f4c579e1-27f8-4317-a696-391e74dc6cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:09.583947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f6e3a-9f08-4a2e-8e54-0c0cfee6b08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:09.584676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f6e3a-9f08-4a2e-8e54-0c0cfee6b08b admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:09.584878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f6e3a-9f08-4a2e-8e54-0c0cfee6b08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:09.585133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f6e3a-9f08-4a2e-8e54-0c0cfee6b08b admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:09.585771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f6e3a-9f08-4a2e-8e54-0c0cfee6b08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:09.604676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f6e3a-9f08-4a2e-8e54-0c0cfee6b08b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:09.655293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1087/2153] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:09:09 2022] POST /identity/v3/users => generated 290 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:09.658868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f6e3a-9f08-4a2e-8e54-0c0cfee6b08b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:09.674332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f6e3a-9f08-4a2e-8e54-0c0cfee6b08b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:09.681521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e483ab-268c-47ad-ad8d-f7a1a35201b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:09.682526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e483ab-268c-47ad-ad8d-f7a1a35201b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:09.682845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e483ab-268c-47ad-ad8d-f7a1a35201b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:09.683223 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e483ab-268c-47ad-ad8d-f7a1a35201b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:09.683820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e483ab-268c-47ad-ad8d-f7a1a35201b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:09.685457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e483ab-268c-47ad-ad8d-f7a1a35201b2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:09.697258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1088/2154] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:09.702404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e483ab-268c-47ad-ad8d-f7a1a35201b2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:09.721200 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c579e1-27f8-4317-a696-391e74dc6cb7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:09.727660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:09.728715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:09.729613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:09.731229 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:09.732176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] PATH_INFO: `/v3/projects/2c39eb313b4f43d0b8e8b60730ed1dd9` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:09.733982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c39eb313b4f43d0b8e8b60730ed1dd9)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:09.737567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:09.738452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e483ab-268c-47ad-ad8d-f7a1a35201b2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:09.747179 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaf7e4-9043-479f-8c49-0c5c8b214a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:09.748650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaf7e4-9043-479f-8c49-0c5c8b214a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:09.749197 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaf7e4-9043-479f-8c49-0c5c8b214a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:09.749556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaf7e4-9043-479f-8c49-0c5c8b214a10 admin admin] PATH_INFO: `/v3/projects/5bf728292e2a4eb2bd1097b01fa6533b/users/636a32ea73fe4987947708bc1862aba2/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:09.778864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaf7e4-9043-479f-8c49-0c5c8b214a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf728292e2a4eb2bd1097b01fa6533b, user_id=636a32ea73fe4987947708bc1862aba2, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:09.782206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaf7e4-9043-479f-8c49-0c5c8b214a10 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:09.810156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1089/2155] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:09 2022] PUT /identity/v3/projects/5bf728292e2a4eb2bd1097b01fa6533b/users/636a32ea73fe4987947708bc1862aba2/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:09:09.810156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaf7e4-9043-479f-8c49-0c5c8b214a10 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:09.865045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c39eb313b4f43d0b8e8b60730ed1dd9', 'request_id': 'req-aa7baad0-b22c-4775-85ec-186cebd2362b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:09.874630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c39eb313b4f43d0b8e8b60730ed1dd9', 'request_id': 'req-aa7baad0-b22c-4775-85ec-186cebd2362b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:09.884924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c39eb313b4f43d0b8e8b60730ed1dd9', 'request_id': 'req-aa7baad0-b22c-4775-85ec-186cebd2362b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:09.928676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] The token cache is being invalidate because project 2c39eb313b4f43d0b8e8b60730ed1dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:09:09.929070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:09.929861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:09.932636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1067/2156] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:09:09 2022] DELETE /identity/v3/projects/2c39eb313b4f43d0b8e8b60730ed1dd9 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:09:09.939557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:09.941371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:10.002055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaf7e4-9043-479f-8c49-0c5c8b214a10 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:10.068612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7baad0-b22c-4775-85ec-186cebd2362b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:10.082834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd376d-00e8-49df-bd76-094016025e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:10.083856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd376d-00e8-49df-bd76-094016025e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:10.084429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd376d-00e8-49df-bd76-094016025e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:10.084806 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd376d-00e8-49df-bd76-094016025e87 admin admin] PATH_INFO: `/v3/users/84e3ac09945f46c7bdd7e15f1d2094d8` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:10.086527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd376d-00e8-49df-bd76-094016025e87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e3ac09945f46c7bdd7e15f1d2094d8)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:10.089466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd376d-00e8-49df-bd76-094016025e87 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:10.112863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:10.125619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac0724-a40d-47e9-b162-61dbe55dee3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:10.126261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac0724-a40d-47e9-b162-61dbe55dee3e admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:10.126546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac0724-a40d-47e9-b162-61dbe55dee3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:10.126806 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac0724-a40d-47e9-b162-61dbe55dee3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:10.127733 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac0724-a40d-47e9-b162-61dbe55dee3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:10.129209 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac0724-a40d-47e9-b162-61dbe55dee3e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:10.159444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1090/2157] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:10.164286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac0724-a40d-47e9-b162-61dbe55dee3e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:10.168110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-53ac0724-a40d-47e9-b162-61dbe55dee3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:10.170623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c6cd376d-00e8-49df-bd76-094016025e87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e3ac09945f46c7bdd7e15f1d2094d8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:10.185827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac0724-a40d-47e9-b162-61dbe55dee3e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:10.190843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f4e9e-89bd-4298-b04b-7ff4a6f3cc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:10.191916 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f4e9e-89bd-4298-b04b-7ff4a6f3cc55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:10.192209 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f4e9e-89bd-4298-b04b-7ff4a6f3cc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:10.192486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f4e9e-89bd-4298-b04b-7ff4a6f3cc55 admin admin] PATH_INFO: `/v3/projects/5bf728292e2a4eb2bd1097b01fa6533b/users/636a32ea73fe4987947708bc1862aba2/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:10.194587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f4e9e-89bd-4298-b04b-7ff4a6f3cc55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf728292e2a4eb2bd1097b01fa6533b, user_id=636a32ea73fe4987947708bc1862aba2, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:10.195873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c6cd376d-00e8-49df-bd76-094016025e87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e3ac09945f46c7bdd7e15f1d2094d8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:10.197411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f4e9e-89bd-4298-b04b-7ff4a6f3cc55 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:10.209512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c6cd376d-00e8-49df-bd76-094016025e87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e3ac09945f46c7bdd7e15f1d2094d8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:10.209743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1091/2158] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:10 2022] PUT /identity/v3/projects/5bf728292e2a4eb2bd1097b01fa6533b/users/636a32ea73fe4987947708bc1862aba2/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:10.210227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c6cd376d-00e8-49df-bd76-094016025e87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e3ac09945f46c7bdd7e15f1d2094d8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:10.220185 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c6cd376d-00e8-49df-bd76-094016025e87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e3ac09945f46c7bdd7e15f1d2094d8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:10.239052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1068/2159] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:09:09 2022] DELETE /identity/v3/users/84e3ac09945f46c7bdd7e15f1d2094d8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:10.845688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f4e9e-89bd-4298-b04b-7ff4a6f3cc55 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:10.847560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-520f4e9e-89bd-4298-b04b-7ff4a6f3cc55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:11.052465 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f4e9e-89bd-4298-b04b-7ff4a6f3cc55 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:11.080555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:11.201306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:11.205398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:11.205398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:11.205398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] PATH_INFO: `/v3/projects/5ee998789a464870ad18f41e14987bba` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:11.210513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ee998789a464870ad18f41e14987bba)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:11.217945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:11.371248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ee998789a464870ad18f41e14987bba', 'request_id': 'req-3be5f982-8398-4b76-a0d4-5c4df33e459c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:11.409437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ee998789a464870ad18f41e14987bba', 'request_id': 'req-3be5f982-8398-4b76-a0d4-5c4df33e459c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:11.449704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ee998789a464870ad18f41e14987bba', 'request_id': 'req-3be5f982-8398-4b76-a0d4-5c4df33e459c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:11.512578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] The token cache is being invalidate because project 5ee998789a464870ad18f41e14987bba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:09:11.513935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:11.518631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3be5f982-8398-4b76-a0d4-5c4df33e459c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:11.525680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1092/2160] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:09:10 2022] DELETE /identity/v3/projects/5ee998789a464870ad18f41e14987bba => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:09:11.530309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c91e3-4fdd-4def-a2c6-cc0370428f26 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:11.531177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c91e3-4fdd-4def-a2c6-cc0370428f26 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:11.531177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c91e3-4fdd-4def-a2c6-cc0370428f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:11.567976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e34c91e3-4fdd-4def-a2c6-cc0370428f26 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:11.569510 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e34c91e3-4fdd-4def-a2c6-cc0370428f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:11.695180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1069/2161] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:11.708260 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84d918-44bf-45a2-b151-8534345b2fb0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:11.724469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84d918-44bf-45a2-b151-8534345b2fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:11.724833 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84d918-44bf-45a2-b151-8534345b2fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:11.766931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3e84d918-44bf-45a2-b151-8534345b2fb0 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:11.768436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3e84d918-44bf-45a2-b151-8534345b2fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:11.780714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1093/2162] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:11.788606 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c91e3-4fdd-4def-a2c6-cc0370428f26 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:11.831000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c91e3-4fdd-4def-a2c6-cc0370428f26 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:11.837350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c08a3-ea03-45fc-86f2-7c666373c0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:11.838217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c08a3-ea03-45fc-86f2-7c666373c0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:11.838690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c08a3-ea03-45fc-86f2-7c666373c0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:11.839201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c08a3-ea03-45fc-86f2-7c666373c0e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:11.839963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c08a3-ea03-45fc-86f2-7c666373c0e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:09:11.840382 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c08a3-ea03-45fc-86f2-7c666373c0e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:11.842143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c08a3-ea03-45fc-86f2-7c666373c0e2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:11.861467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1070/2163] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 17:09:11.875171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84d918-44bf-45a2-b151-8534345b2fb0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:11.910053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84d918-44bf-45a2-b151-8534345b2fb0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:11.935899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638868b-8bb8-4860-bc37-9295787fd682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:11.936968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638868b-8bb8-4860-bc37-9295787fd682 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:11.937466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638868b-8bb8-4860-bc37-9295787fd682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:11.937954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638868b-8bb8-4860-bc37-9295787fd682 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:11.939262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638868b-8bb8-4860-bc37-9295787fd682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:11.942467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638868b-8bb8-4860-bc37-9295787fd682 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:12.011027 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c08a3-ea03-45fc-86f2-7c666373c0e2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:12.019502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1094/2164] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:09:11 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:12.074205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c08a3-ea03-45fc-86f2-7c666373c0e2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:12.310880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47dd4b-3470-4e5c-b07c-91511664c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:12.311660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47dd4b-3470-4e5c-b07c-91511664c174 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:12.311938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47dd4b-3470-4e5c-b07c-91511664c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:12.312803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47dd4b-3470-4e5c-b07c-91511664c174 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:12.313523 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47dd4b-3470-4e5c-b07c-91511664c174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:12.315926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47dd4b-3470-4e5c-b07c-91511664c174 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:12.383484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1071/2165] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:09:12 2022] POST /identity/v3/users => generated 294 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:09:12.387950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638868b-8bb8-4860-bc37-9295787fd682 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:12.416414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638868b-8bb8-4860-bc37-9295787fd682 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:12.473803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb32c4-c135-4627-8957-ad9aa1ea9fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:12.474990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb32c4-c135-4627-8957-ad9aa1ea9fde admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:12.479382 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb32c4-c135-4627-8957-ad9aa1ea9fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:12.479872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb32c4-c135-4627-8957-ad9aa1ea9fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:12.483397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb32c4-c135-4627-8957-ad9aa1ea9fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:12.495222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb32c4-c135-4627-8957-ad9aa1ea9fde admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:12.522105 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1095/2166] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:09:12.533560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47dd4b-3470-4e5c-b07c-91511664c174 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:12.586002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47dd4b-3470-4e5c-b07c-91511664c174 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:12.603581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6f9e1-dfc2-4213-aa2b-8314f842c280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:12.606622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6f9e1-dfc2-4213-aa2b-8314f842c280 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:12.609726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6f9e1-dfc2-4213-aa2b-8314f842c280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:12.610421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6f9e1-dfc2-4213-aa2b-8314f842c280 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:12.611970 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6f9e1-dfc2-4213-aa2b-8314f842c280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:12.621423 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6f9e1-dfc2-4213-aa2b-8314f842c280 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:12.639092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:12.657460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1072/2167] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:12.661718 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb32c4-c135-4627-8957-ad9aa1ea9fde admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:12.699706 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb32c4-c135-4627-8957-ad9aa1ea9fde admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:12.710427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e900273-a30f-48df-a388-a78e3d73dbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:12.711986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e900273-a30f-48df-a388-a78e3d73dbcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:12.712488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e900273-a30f-48df-a388-a78e3d73dbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:12.713036 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e900273-a30f-48df-a388-a78e3d73dbcd admin admin] PATH_INFO: `/v3/projects/528096c737fe4e5aad13caadcf0e957a/users/97f13a1b652c413bb8ba893b34e3517a/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:12.752896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e900273-a30f-48df-a388-a78e3d73dbcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=528096c737fe4e5aad13caadcf0e957a, user_id=97f13a1b652c413bb8ba893b34e3517a, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:12.761703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e900273-a30f-48df-a388-a78e3d73dbcd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:12.792332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1096/2168] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:12 2022] PUT /identity/v3/projects/528096c737fe4e5aad13caadcf0e957a/users/97f13a1b652c413bb8ba893b34e3517a/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:12.805214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be7f31-348f-4b14-bc1c-5e3799ff195d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:12.806356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be7f31-348f-4b14-bc1c-5e3799ff195d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:12.806785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be7f31-348f-4b14-bc1c-5e3799ff195d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:12.848959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-15be7f31-348f-4b14-bc1c-5e3799ff195d None None] MFA Rules not processed for user `97f13a1b652c413bb8ba893b34e3517a`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:12.850587 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-15be7f31-348f-4b14-bc1c-5e3799ff195d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:12.975139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1073/2169] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:12.980136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e900273-a30f-48df-a388-a78e3d73dbcd admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:13.060690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:13.108499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e900273-a30f-48df-a388-a78e3d73dbcd admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:13.114926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f19bb-6a38-4c69-81c5-1b68bba2ff6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:13.116001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f19bb-6a38-4c69-81c5-1b68bba2ff6d admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:13.116842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f19bb-6a38-4c69-81c5-1b68bba2ff6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:13.117300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f19bb-6a38-4c69-81c5-1b68bba2ff6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:13.118308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f19bb-6a38-4c69-81c5-1b68bba2ff6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:13.121003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f19bb-6a38-4c69-81c5-1b68bba2ff6d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:13.173275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1097/2170] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:09:12 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:13.177326 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be7f31-348f-4b14-bc1c-5e3799ff195d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:13.198077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be7f31-348f-4b14-bc1c-5e3799ff195d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:13.298932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276b666-a0ff-43aa-a7ef-2cc805b8aa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:13.300273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276b666-a0ff-43aa-a7ef-2cc805b8aa61 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:13.300630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276b666-a0ff-43aa-a7ef-2cc805b8aa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:13.300845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276b666-a0ff-43aa-a7ef-2cc805b8aa61 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:13.302355 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276b666-a0ff-43aa-a7ef-2cc805b8aa61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:13.306623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276b666-a0ff-43aa-a7ef-2cc805b8aa61 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:13.347571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1074/2171] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:09:13 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:13.349058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f19bb-6a38-4c69-81c5-1b68bba2ff6d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:13.378776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f19bb-6a38-4c69-81c5-1b68bba2ff6d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:13.396225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1262034-ef22-43c2-bd61-fc7d2d4c5ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:13.398775 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1262034-ef22-43c2-bd61-fc7d2d4c5ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:13.400403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1262034-ef22-43c2-bd61-fc7d2d4c5ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:13.402252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1262034-ef22-43c2-bd61-fc7d2d4c5ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:13.402985 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1262034-ef22-43c2-bd61-fc7d2d4c5ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:13.411653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1262034-ef22-43c2-bd61-fc7d2d4c5ef9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:13.428568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1098/2172] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:09:13.435000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276b666-a0ff-43aa-a7ef-2cc805b8aa61 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:13.451254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276b666-a0ff-43aa-a7ef-2cc805b8aa61 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:13.458523 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4893ace-238b-40d3-9c24-aa34d4b2fd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:13.459962 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4893ace-238b-40d3-9c24-aa34d4b2fd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:13.460244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4893ace-238b-40d3-9c24-aa34d4b2fd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:13.460542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4893ace-238b-40d3-9c24-aa34d4b2fd2c admin admin] PATH_INFO: `/v3/projects/f0ef1a3ca4ec4671bde5ad7bf2d92aa2/users/ebf652b3978a4ed78963c24a8cc7ca89/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:13.485759 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4893ace-238b-40d3-9c24-aa34d4b2fd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ef1a3ca4ec4671bde5ad7bf2d92aa2, user_id=ebf652b3978a4ed78963c24a8cc7ca89, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:13.489596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4893ace-238b-40d3-9c24-aa34d4b2fd2c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:13.513356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1075/2173] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:13 2022] PUT /identity/v3/projects/f0ef1a3ca4ec4671bde5ad7bf2d92aa2/users/ebf652b3978a4ed78963c24a8cc7ca89/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:13.521350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1262034-ef22-43c2-bd61-fc7d2d4c5ef9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:13.544558 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1262034-ef22-43c2-bd61-fc7d2d4c5ef9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:13.698661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83dc77-5afa-4505-94f7-34ba071cb0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:13.699681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83dc77-5afa-4505-94f7-34ba071cb0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:13.700116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83dc77-5afa-4505-94f7-34ba071cb0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:13.700560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83dc77-5afa-4505-94f7-34ba071cb0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:13.701411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83dc77-5afa-4505-94f7-34ba071cb0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:13.704072 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83dc77-5afa-4505-94f7-34ba071cb0d3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:13.731378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1099/2174] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:13.753227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4893ace-238b-40d3-9c24-aa34d4b2fd2c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:13.782833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4893ace-238b-40d3-9c24-aa34d4b2fd2c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:13.792331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17c3c2-432d-471b-915c-53e6f77e15d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:13.793598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17c3c2-432d-471b-915c-53e6f77e15d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:13.794180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17c3c2-432d-471b-915c-53e6f77e15d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:13.794889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17c3c2-432d-471b-915c-53e6f77e15d7 admin admin] PATH_INFO: `/v3/projects/f0ef1a3ca4ec4671bde5ad7bf2d92aa2/users/ebf652b3978a4ed78963c24a8cc7ca89/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:13.799120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17c3c2-432d-471b-915c-53e6f77e15d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ef1a3ca4ec4671bde5ad7bf2d92aa2, user_id=ebf652b3978a4ed78963c24a8cc7ca89, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:13.805018 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17c3c2-432d-471b-915c-53e6f77e15d7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:13.831227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1076/2175] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:13 2022] PUT /identity/v3/projects/f0ef1a3ca4ec4671bde5ad7bf2d92aa2/users/ebf652b3978a4ed78963c24a8cc7ca89/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:13.856644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d946fcd-0ba0-4ad8-9cfa-a019797397b6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:13.856821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d946fcd-0ba0-4ad8-9cfa-a019797397b6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:13.857297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d946fcd-0ba0-4ad8-9cfa-a019797397b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:13.934014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2d946fcd-0ba0-4ad8-9cfa-a019797397b6 None None] MFA Rules not processed for user `ebf652b3978a4ed78963c24a8cc7ca89`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:13.938138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2d946fcd-0ba0-4ad8-9cfa-a019797397b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:14.062594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1100/2176] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:14.074718 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17c3c2-432d-471b-915c-53e6f77e15d7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:14.078580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5f17c3c2-432d-471b-915c-53e6f77e15d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:14.176556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:14.202315 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17c3c2-432d-471b-915c-53e6f77e15d7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:14.206328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955eb85-d0a3-4e4c-9fd5-5f9b3fe6613c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:14.207055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955eb85-d0a3-4e4c-9fd5-5f9b3fe6613c service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:14.207240 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955eb85-d0a3-4e4c-9fd5-5f9b3fe6613c service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:14.207511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955eb85-d0a3-4e4c-9fd5-5f9b3fe6613c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:14.219781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955eb85-d0a3-4e4c-9fd5-5f9b3fe6613c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:14.224772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955eb85-d0a3-4e4c-9fd5-5f9b3fe6613c service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:14.251636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1077/2177] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:14.326595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d946fcd-0ba0-4ad8-9cfa-a019797397b6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:14.451874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:14.495111 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d946fcd-0ba0-4ad8-9cfa-a019797397b6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:14.501277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:14.501977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:14.502350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:14.502629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] PATH_INFO: `/v3/users/97f13a1b652c413bb8ba893b34e3517a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:14.504125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f13a1b652c413bb8ba893b34e3517a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:14.506366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:14.690888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f13a1b652c413bb8ba893b34e3517a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:14.710599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f13a1b652c413bb8ba893b34e3517a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:14.750701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f13a1b652c413bb8ba893b34e3517a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:14.789191 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f13a1b652c413bb8ba893b34e3517a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:14.805919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f13a1b652c413bb8ba893b34e3517a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:14.817566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1101/2178] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:09:14 2022] DELETE /identity/v3/users/97f13a1b652c413bb8ba893b34e3517a => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:14.826135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473493b-9ad1-4910-b522-8c4a45268f56 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:14.827074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473493b-9ad1-4910-b522-8c4a45268f56 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:14.827684 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473493b-9ad1-4910-b522-8c4a45268f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:14.862739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9473493b-9ad1-4910-b522-8c4a45268f56 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:14.864858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9473493b-9ad1-4910-b522-8c4a45268f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:14.960512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1078/2179] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:14.972310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:14.973785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:15.049680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef49a9e-87a2-4222-aa3c-091629c3a59a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:15.055330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb660fb-b4bf-42fb-84ca-a0bc385a8033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:15.056728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb660fb-b4bf-42fb-84ca-a0bc385a8033 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:15.057143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb660fb-b4bf-42fb-84ca-a0bc385a8033 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:15.057557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb660fb-b4bf-42fb-84ca-a0bc385a8033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:15.070911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb660fb-b4bf-42fb-84ca-a0bc385a8033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:15.072991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb660fb-b4bf-42fb-84ca-a0bc385a8033 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:15.084467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1102/2180] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:15.495850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473493b-9ad1-4910-b522-8c4a45268f56 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:15.497322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9473493b-9ad1-4910-b522-8c4a45268f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:15.517137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473493b-9ad1-4910-b522-8c4a45268f56 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:15.533556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:15.535374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:15.536006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:15.536616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] PATH_INFO: `/v3/projects/528096c737fe4e5aad13caadcf0e957a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:15.539412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=528096c737fe4e5aad13caadcf0e957a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:15.541898 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:15.628771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528096c737fe4e5aad13caadcf0e957a', 'request_id': 'req-b84832c1-4902-4706-8fb5-dcb07012f675'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:15.641948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528096c737fe4e5aad13caadcf0e957a', 'request_id': 'req-b84832c1-4902-4706-8fb5-dcb07012f675'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:15.651719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528096c737fe4e5aad13caadcf0e957a', 'request_id': 'req-b84832c1-4902-4706-8fb5-dcb07012f675'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:15.673173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] The token cache is being invalidate because project 528096c737fe4e5aad13caadcf0e957a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:09:15.673845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:15.675550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:15.678557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1079/2181] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:09:15 2022] DELETE /identity/v3/projects/528096c737fe4e5aad13caadcf0e957a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:15.682610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb660fb-b4bf-42fb-84ca-a0bc385a8033 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:15.699851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb660fb-b4bf-42fb-84ca-a0bc385a8033 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:15.794411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4ccfe-bfcf-4115-ba22-2c1b94773fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:15.795585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4ccfe-bfcf-4115-ba22-2c1b94773fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:15.795585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4ccfe-bfcf-4115-ba22-2c1b94773fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:15.797196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4ccfe-bfcf-4115-ba22-2c1b94773fcd admin admin] PATH_INFO: `/v3/users/ebf652b3978a4ed78963c24a8cc7ca89` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:15.801585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4ccfe-bfcf-4115-ba22-2c1b94773fcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebf652b3978a4ed78963c24a8cc7ca89)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:15.806449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4ccfe-bfcf-4115-ba22-2c1b94773fcd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:15.809099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:15.892397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4ff4ccfe-bfcf-4115-ba22-2c1b94773fcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebf652b3978a4ed78963c24a8cc7ca89'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:15.937042 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4ff4ccfe-bfcf-4115-ba22-2c1b94773fcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebf652b3978a4ed78963c24a8cc7ca89'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:15.970138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4ff4ccfe-bfcf-4115-ba22-2c1b94773fcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebf652b3978a4ed78963c24a8cc7ca89'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:15.994584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4ff4ccfe-bfcf-4115-ba22-2c1b94773fcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebf652b3978a4ed78963c24a8cc7ca89'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:15.995281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4ff4ccfe-bfcf-4115-ba22-2c1b94773fcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebf652b3978a4ed78963c24a8cc7ca89'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:16.005375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1103/2182] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:09:15 2022] DELETE /identity/v3/users/ebf652b3978a4ed78963c24a8cc7ca89 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:16.562250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:16.564571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:16.590012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84832c1-4902-4706-8fb5-dcb07012f675 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:16.688658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:16.689450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:16.689565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:16.689830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] PATH_INFO: `/v3/projects/f0ef1a3ca4ec4671bde5ad7bf2d92aa2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:16.691002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ef1a3ca4ec4671bde5ad7bf2d92aa2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:16.692933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:16.708452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:16.779896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ef1a3ca4ec4671bde5ad7bf2d92aa2', 'request_id': 'req-630d3768-9929-4dc5-938e-d5354e1bd502'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:16.786580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ef1a3ca4ec4671bde5ad7bf2d92aa2', 'request_id': 'req-630d3768-9929-4dc5-938e-d5354e1bd502'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:16.793405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ef1a3ca4ec4671bde5ad7bf2d92aa2', 'request_id': 'req-630d3768-9929-4dc5-938e-d5354e1bd502'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:16.808430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] The token cache is being invalidate because project f0ef1a3ca4ec4671bde5ad7bf2d92aa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:09:16.808719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:16.809364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-630d3768-9929-4dc5-938e-d5354e1bd502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:16.811196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1080/2183] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:09:16 2022] DELETE /identity/v3/projects/f0ef1a3ca4ec4671bde5ad7bf2d92aa2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:16.816511 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb03926-7d29-4826-8d65-d0ce3af40b64 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:16.816998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb03926-7d29-4826-8d65-d0ce3af40b64 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:16.817081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb03926-7d29-4826-8d65-d0ce3af40b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:16.842389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-feb03926-7d29-4826-8d65-d0ce3af40b64 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:16.843654 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-feb03926-7d29-4826-8d65-d0ce3af40b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:16.911376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1104/2184] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:16.919282 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d32da-fac9-4c8b-9769-46a9a09b6759 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:16.919566 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d32da-fac9-4c8b-9769-46a9a09b6759 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:16.919846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d32da-fac9-4c8b-9769-46a9a09b6759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:16.944220 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-059d32da-fac9-4c8b-9769-46a9a09b6759 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:16.945384 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-059d32da-fac9-4c8b-9769-46a9a09b6759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:16.954685 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1081/2185] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:16.958922 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb03926-7d29-4826-8d65-d0ce3af40b64 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:16.969076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb03926-7d29-4826-8d65-d0ce3af40b64 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:16.973812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020361e9-b044-4718-9d08-abb3c678df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:16.974519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020361e9-b044-4718-9d08-abb3c678df27 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:16.974865 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020361e9-b044-4718-9d08-abb3c678df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:16.975301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020361e9-b044-4718-9d08-abb3c678df27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:16.976016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020361e9-b044-4718-9d08-abb3c678df27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:09:16.976419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020361e9-b044-4718-9d08-abb3c678df27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:16.978715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020361e9-b044-4718-9d08-abb3c678df27 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:16.996885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1105/2186] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 17:09:17.000925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d32da-fac9-4c8b-9769-46a9a09b6759 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:17.009473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d32da-fac9-4c8b-9769-46a9a09b6759 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:17.017534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ca9e2-ec2f-4921-a1d4-96e91ab81a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:17.017534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ca9e2-ec2f-4921-a1d4-96e91ab81a06 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.017534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ca9e2-ec2f-4921-a1d4-96e91ab81a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.017534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ca9e2-ec2f-4921-a1d4-96e91ab81a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.017975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ca9e2-ec2f-4921-a1d4-96e91ab81a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:17.019742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ca9e2-ec2f-4921-a1d4-96e91ab81a06 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:17.052105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1082/2187] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:09:16 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:17.056992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020361e9-b044-4718-9d08-abb3c678df27 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:17.070278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020361e9-b044-4718-9d08-abb3c678df27 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:17.141454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67866ed3-4cd2-4fc9-a7c4-140f979b5881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:17.142188 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67866ed3-4cd2-4fc9-a7c4-140f979b5881 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.142475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67866ed3-4cd2-4fc9-a7c4-140f979b5881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.142751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67866ed3-4cd2-4fc9-a7c4-140f979b5881 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.143439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67866ed3-4cd2-4fc9-a7c4-140f979b5881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:17.145457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67866ed3-4cd2-4fc9-a7c4-140f979b5881 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:17.157127 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:17.169801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1106/2188] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:09:17 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:17.174042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ca9e2-ec2f-4921-a1d4-96e91ab81a06 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:17.185722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ca9e2-ec2f-4921-a1d4-96e91ab81a06 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:17.190255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f37488-b9ce-495f-98a3-e41d509ea201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:17.190856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f37488-b9ce-495f-98a3-e41d509ea201 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.191168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f37488-b9ce-495f-98a3-e41d509ea201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.191442 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f37488-b9ce-495f-98a3-e41d509ea201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.192086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f37488-b9ce-495f-98a3-e41d509ea201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:17.196891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f37488-b9ce-495f-98a3-e41d509ea201 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:17.204761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1083/2189] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:17.208995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67866ed3-4cd2-4fc9-a7c4-140f979b5881 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:17.221225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67866ed3-4cd2-4fc9-a7c4-140f979b5881 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:17.226286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14963b-8146-4e41-b26a-5acb5b91e0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:17.228765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14963b-8146-4e41-b26a-5acb5b91e0ad admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.228765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14963b-8146-4e41-b26a-5acb5b91e0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.228765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14963b-8146-4e41-b26a-5acb5b91e0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.228765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14963b-8146-4e41-b26a-5acb5b91e0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:17.234116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14963b-8146-4e41-b26a-5acb5b91e0ad admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:17.259306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1107/2190] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:17.260969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f37488-b9ce-495f-98a3-e41d509ea201 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:17.272398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f37488-b9ce-495f-98a3-e41d509ea201 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:17.278032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da0661-aa62-40c1-9a79-c7fc4d5c1a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:17.278888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da0661-aa62-40c1-9a79-c7fc4d5c1a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.279291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da0661-aa62-40c1-9a79-c7fc4d5c1a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.279634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da0661-aa62-40c1-9a79-c7fc4d5c1a6d admin admin] PATH_INFO: `/v3/projects/50a2613e2b0c4ee18488ee04d6aba3b2/users/17701b233d83462d8b57755048780c3b/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.297738 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da0661-aa62-40c1-9a79-c7fc4d5c1a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a2613e2b0c4ee18488ee04d6aba3b2, user_id=17701b233d83462d8b57755048780c3b, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:17.300794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da0661-aa62-40c1-9a79-c7fc4d5c1a6d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:17.317843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1084/2191] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:17 2022] PUT /identity/v3/projects/50a2613e2b0c4ee18488ee04d6aba3b2/users/17701b233d83462d8b57755048780c3b/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:17.329636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5878358-636f-4402-982c-2e5252b7d407 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.330038 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5878358-636f-4402-982c-2e5252b7d407 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.330384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5878358-636f-4402-982c-2e5252b7d407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.366444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a5878358-636f-4402-982c-2e5252b7d407 None None] MFA Rules not processed for user `17701b233d83462d8b57755048780c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:17.367740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a5878358-636f-4402-982c-2e5252b7d407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:17.450032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1108/2192] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:17.454728 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da0661-aa62-40c1-9a79-c7fc4d5c1a6d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:17.529107 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da0661-aa62-40c1-9a79-c7fc4d5c1a6d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:17.534800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4900ef-86c4-40b5-88c9-697418d2af72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:17.535462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4900ef-86c4-40b5-88c9-697418d2af72 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.535757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4900ef-86c4-40b5-88c9-697418d2af72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.536030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4900ef-86c4-40b5-88c9-697418d2af72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.536692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4900ef-86c4-40b5-88c9-697418d2af72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:17.538619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4900ef-86c4-40b5-88c9-697418d2af72 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:17.570027 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1085/2193] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:09:17 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:17.574773 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5878358-636f-4402-982c-2e5252b7d407 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:17.588731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5878358-636f-4402-982c-2e5252b7d407 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:17.658365 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e2846-4769-444d-846e-7d9b0fb3f588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:17.659141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e2846-4769-444d-846e-7d9b0fb3f588 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.659365 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e2846-4769-444d-846e-7d9b0fb3f588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.659627 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e2846-4769-444d-846e-7d9b0fb3f588 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.660295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e2846-4769-444d-846e-7d9b0fb3f588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:17.662280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e2846-4769-444d-846e-7d9b0fb3f588 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:17.695167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1109/2194] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:09:17 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:17.697740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4900ef-86c4-40b5-88c9-697418d2af72 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:17.711233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4900ef-86c4-40b5-88c9-697418d2af72 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:17.715887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8815dc-e4ad-4e7e-aff4-fff107512b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:17.716490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8815dc-e4ad-4e7e-aff4-fff107512b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.716765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8815dc-e4ad-4e7e-aff4-fff107512b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.717063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8815dc-e4ad-4e7e-aff4-fff107512b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.717699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8815dc-e4ad-4e7e-aff4-fff107512b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:17.719331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8815dc-e4ad-4e7e-aff4-fff107512b17 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:17.732871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1086/2195] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:17.737120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e2846-4769-444d-846e-7d9b0fb3f588 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:17.749908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e2846-4769-444d-846e-7d9b0fb3f588 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:17.758385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f4bde-2512-4168-93f1-a8143a88b2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:17.763279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f4bde-2512-4168-93f1-a8143a88b2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.763279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f4bde-2512-4168-93f1-a8143a88b2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.763279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f4bde-2512-4168-93f1-a8143a88b2b7 admin admin] PATH_INFO: `/v3/projects/f42e86ea1b0e4813b2ae6a2c2fd13be9/users/c6641f93e65144ac9b29efacedecd197/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.790731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f4bde-2512-4168-93f1-a8143a88b2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42e86ea1b0e4813b2ae6a2c2fd13be9, user_id=c6641f93e65144ac9b29efacedecd197, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:17.793449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f4bde-2512-4168-93f1-a8143a88b2b7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:17.810048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1110/2196] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:17 2022] PUT /identity/v3/projects/f42e86ea1b0e4813b2ae6a2c2fd13be9/users/c6641f93e65144ac9b29efacedecd197/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:17.814339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8815dc-e4ad-4e7e-aff4-fff107512b17 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:17.826435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8815dc-e4ad-4e7e-aff4-fff107512b17 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:17.835075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:17.900465 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f6eff-00d4-4590-bdf1-4cae0951c8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:17.901142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f6eff-00d4-4590-bdf1-4cae0951c8fd admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.901976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f6eff-00d4-4590-bdf1-4cae0951c8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.902363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f6eff-00d4-4590-bdf1-4cae0951c8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.903412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f6eff-00d4-4590-bdf1-4cae0951c8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:17.905276 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f6eff-00d4-4590-bdf1-4cae0951c8fd admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:17.919135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1087/2197] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:17.922525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f4bde-2512-4168-93f1-a8143a88b2b7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:17.934635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f4bde-2512-4168-93f1-a8143a88b2b7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:17.941196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ddd0c0-7337-467a-88c8-b44aedb4e84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:17.942330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ddd0c0-7337-467a-88c8-b44aedb4e84f admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.942330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ddd0c0-7337-467a-88c8-b44aedb4e84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.942550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ddd0c0-7337-467a-88c8-b44aedb4e84f admin admin] PATH_INFO: `/v3/projects/f42e86ea1b0e4813b2ae6a2c2fd13be9/users/c6641f93e65144ac9b29efacedecd197/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:17.944618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ddd0c0-7337-467a-88c8-b44aedb4e84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42e86ea1b0e4813b2ae6a2c2fd13be9, user_id=c6641f93e65144ac9b29efacedecd197, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:17.947898 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ddd0c0-7337-467a-88c8-b44aedb4e84f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:17.971462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1111/2198] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:17 2022] PUT /identity/v3/projects/f42e86ea1b0e4813b2ae6a2c2fd13be9/users/c6641f93e65144ac9b29efacedecd197/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:17.978783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77b3d3-3593-4ee2-a8ec-6830012aff6a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:17.979103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77b3d3-3593-4ee2-a8ec-6830012aff6a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:17.979378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77b3d3-3593-4ee2-a8ec-6830012aff6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:18.021107 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ad77b3d3-3593-4ee2-a8ec-6830012aff6a None None] MFA Rules not processed for user `c6641f93e65144ac9b29efacedecd197`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:18.022409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ad77b3d3-3593-4ee2-a8ec-6830012aff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:18.130601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1088/2199] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:18.141809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ddd0c0-7337-467a-88c8-b44aedb4e84f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:18.143590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-23ddd0c0-7337-467a-88c8-b44aedb4e84f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:18.196011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:18.229877 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ddd0c0-7337-467a-88c8-b44aedb4e84f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:18.233558 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee074aa-722c-4d93-83cc-b02b3c43cfcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:18.234160 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee074aa-722c-4d93-83cc-b02b3c43cfcc service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:18.234435 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee074aa-722c-4d93-83cc-b02b3c43cfcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:18.234701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee074aa-722c-4d93-83cc-b02b3c43cfcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:18.297450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee074aa-722c-4d93-83cc-b02b3c43cfcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:18.299029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee074aa-722c-4d93-83cc-b02b3c43cfcc service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:18.308841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1112/2200] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:20.264542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f9553-fb28-4d58-a386-db28ce73b739 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:20.265149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f9553-fb28-4d58-a386-db28ce73b739 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:20.265149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f9553-fb28-4d58-a386-db28ce73b739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:20.297878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4d1f9553-fb28-4d58-a386-db28ce73b739 None None] MFA Rules not processed for user `636a32ea73fe4987947708bc1862aba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:20.299155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4d1f9553-fb28-4d58-a386-db28ce73b739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:20.426107 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1089/2201] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:20.442628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee074aa-722c-4d93-83cc-b02b3c43cfcc service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:20.444365 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3ee074aa-722c-4d93-83cc-b02b3c43cfcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:20.545672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee074aa-722c-4d93-83cc-b02b3c43cfcc service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:20.551674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287ffde-33bc-4f20-a26c-c9b7b0109a97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:20.552441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ffde-33bc-4f20-a26c-c9b7b0109a97 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:20.552719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ffde-33bc-4f20-a26c-c9b7b0109a97 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:20.552991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ffde-33bc-4f20-a26c-c9b7b0109a97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:20.557448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e287ffde-33bc-4f20-a26c-c9b7b0109a97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:20.628736 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287ffde-33bc-4f20-a26c-c9b7b0109a97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:20.630341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287ffde-33bc-4f20-a26c-c9b7b0109a97 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:20.647008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1113/2202] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:21.901315 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc666745-bfd0-4888-bbf4-4c56b3202c62 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:21.901757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc666745-bfd0-4888-bbf4-4c56b3202c62 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:21.902090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc666745-bfd0-4888-bbf4-4c56b3202c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:21.945344 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-cc666745-bfd0-4888-bbf4-4c56b3202c62 None None] MFA Rules not processed for user `0365ab1bcfd54946b0b92f8fe9697128`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:21.947517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cc666745-bfd0-4888-bbf4-4c56b3202c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:21.970506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1090/2203] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:09:26.250193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287ffde-33bc-4f20-a26c-c9b7b0109a97 service nova] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:26.260154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287ffde-33bc-4f20-a26c-c9b7b0109a97 service nova] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:26.263730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70a9bb-5cf1-491a-b3bb-de093db2cdea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:26.264708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70a9bb-5cf1-491a-b3bb-de093db2cdea service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:26.264708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70a9bb-5cf1-491a-b3bb-de093db2cdea service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:26.265083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70a9bb-5cf1-491a-b3bb-de093db2cdea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:26.275404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70a9bb-5cf1-491a-b3bb-de093db2cdea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:26.276882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70a9bb-5cf1-491a-b3bb-de093db2cdea service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:26.287065 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1114/2204] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:31.936265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:32.302395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc666745-bfd0-4888-bbf4-4c56b3202c62 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:32.366644 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc666745-bfd0-4888-bbf4-4c56b3202c62 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:32.371271 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:32.371907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:32.372188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:32.372461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] PATH_INFO: `/v3/users/17701b233d83462d8b57755048780c3b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:32.373610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] RBAC: Authorizing `identity:delete_user(user_id=17701b233d83462d8b57755048780c3b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:32.375582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:32.441582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17701b233d83462d8b57755048780c3b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:32.459707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17701b233d83462d8b57755048780c3b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:32.468309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17701b233d83462d8b57755048780c3b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:32.469740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17701b233d83462d8b57755048780c3b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:32.478452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17701b233d83462d8b57755048780c3b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:32.487196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1091/2205] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:09:32 2022] DELETE /identity/v3/users/17701b233d83462d8b57755048780c3b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:32.491857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22071b56-67b2-4707-8051-e65eb28c60aa None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:32.492371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22071b56-67b2-4707-8051-e65eb28c60aa None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:32.492495 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22071b56-67b2-4707-8051-e65eb28c60aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:32.500566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:32.511883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-22071b56-67b2-4707-8051-e65eb28c60aa None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:32.513087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-22071b56-67b2-4707-8051-e65eb28c60aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:32.579859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1115/2206] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:09:32.590129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:32.591684 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:32.666757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33d64d-b445-4e98-8a11-40607e1ea79e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:32.670843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56940d7-0594-4ec7-86c7-4c0383bfc2c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:32.671594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56940d7-0594-4ec7-86c7-4c0383bfc2c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:32.671886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56940d7-0594-4ec7-86c7-4c0383bfc2c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:32.672314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56940d7-0594-4ec7-86c7-4c0383bfc2c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:32.684037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56940d7-0594-4ec7-86c7-4c0383bfc2c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:32.686395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56940d7-0594-4ec7-86c7-4c0383bfc2c0 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:32.706855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1092/2207] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:33.048075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22071b56-67b2-4707-8051-e65eb28c60aa None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:33.050599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-22071b56-67b2-4707-8051-e65eb28c60aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:33.069473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22071b56-67b2-4707-8051-e65eb28c60aa None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:33.076051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:33.076963 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:33.077403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:33.079171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] PATH_INFO: `/v3/projects/50a2613e2b0c4ee18488ee04d6aba3b2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:33.082093 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a2613e2b0c4ee18488ee04d6aba3b2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:33.083991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:33.146324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a2613e2b0c4ee18488ee04d6aba3b2', 'request_id': 'req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.154849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a2613e2b0c4ee18488ee04d6aba3b2', 'request_id': 'req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.161642 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a2613e2b0c4ee18488ee04d6aba3b2', 'request_id': 'req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.178723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] The token cache is being invalidate because project 50a2613e2b0c4ee18488ee04d6aba3b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:09:33.179055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.180792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.182936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1116/2208] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:09:33 2022] DELETE /identity/v3/projects/50a2613e2b0c4ee18488ee04d6aba3b2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:33.187249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56940d7-0594-4ec7-86c7-4c0383bfc2c0 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:33.210024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56940d7-0594-4ec7-86c7-4c0383bfc2c0 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:33.232228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:33.285422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee4972-9aa2-4084-9c98-4d07ef9d905f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:33.286332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4972-9aa2-4084-9c98-4d07ef9d905f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:33.286645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4972-9aa2-4084-9c98-4d07ef9d905f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:33.287029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4972-9aa2-4084-9c98-4d07ef9d905f admin admin] PATH_INFO: `/v3/users/c6641f93e65144ac9b29efacedecd197` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:33.288798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee4972-9aa2-4084-9c98-4d07ef9d905f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6641f93e65144ac9b29efacedecd197)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:33.290647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee4972-9aa2-4084-9c98-4d07ef9d905f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:33.349932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1aee4972-9aa2-4084-9c98-4d07ef9d905f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6641f93e65144ac9b29efacedecd197'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.362855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1aee4972-9aa2-4084-9c98-4d07ef9d905f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6641f93e65144ac9b29efacedecd197'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.370497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1aee4972-9aa2-4084-9c98-4d07ef9d905f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6641f93e65144ac9b29efacedecd197'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.371303 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1aee4972-9aa2-4084-9c98-4d07ef9d905f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6641f93e65144ac9b29efacedecd197'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.379721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1aee4972-9aa2-4084-9c98-4d07ef9d905f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6641f93e65144ac9b29efacedecd197'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.387006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1093/2209] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:09:33 2022] DELETE /identity/v3/users/c6641f93e65144ac9b29efacedecd197 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:33.690719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:33.692279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:33.706717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9fde0-23e1-4e3f-8954-4bb1cc6efc59 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:33.769843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:33.770491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:33.770772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:33.771080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] PATH_INFO: `/v3/projects/f42e86ea1b0e4813b2ae6a2c2fd13be9` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:33.772217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42e86ea1b0e4813b2ae6a2c2fd13be9)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:33.774828 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:33.813181 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:33.828453 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42e86ea1b0e4813b2ae6a2c2fd13be9', 'request_id': 'req-625d3183-dd12-447d-b9c8-62ebf44aca52'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.835083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42e86ea1b0e4813b2ae6a2c2fd13be9', 'request_id': 'req-625d3183-dd12-447d-b9c8-62ebf44aca52'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.841528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42e86ea1b0e4813b2ae6a2c2fd13be9', 'request_id': 'req-625d3183-dd12-447d-b9c8-62ebf44aca52'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.855018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] The token cache is being invalidate because project f42e86ea1b0e4813b2ae6a2c2fd13be9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:09:33.855306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.855952 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-625d3183-dd12-447d-b9c8-62ebf44aca52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:33.857426 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1117/2210] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:09:33 2022] DELETE /identity/v3/projects/f42e86ea1b0e4813b2ae6a2c2fd13be9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:33.863458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5d912-d42a-49cf-b42a-b9eb4a08b2f0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:33.863765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5d912-d42a-49cf-b42a-b9eb4a08b2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:33.864050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5d912-d42a-49cf-b42a-b9eb4a08b2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:33.885105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-dde5d912-d42a-49cf-b42a-b9eb4a08b2f0 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:33.886341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-dde5d912-d42a-49cf-b42a-b9eb4a08b2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:33.948053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1094/2211] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:33.956473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e774db2-cb99-4287-aabc-e1557b479ca4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:33.956762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e774db2-cb99-4287-aabc-e1557b479ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:33.957034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e774db2-cb99-4287-aabc-e1557b479ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:33.978619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-9e774db2-cb99-4287-aabc-e1557b479ca4 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:33.979907 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9e774db2-cb99-4287-aabc-e1557b479ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:33.988014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1118/2212] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:33.995090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5d912-d42a-49cf-b42a-b9eb4a08b2f0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:34.006234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5d912-d42a-49cf-b42a-b9eb4a08b2f0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:34.011483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4479caa-d0c8-4335-ac65-3dff157d10c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:34.012108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4479caa-d0c8-4335-ac65-3dff157d10c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:34.012391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4479caa-d0c8-4335-ac65-3dff157d10c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:34.012902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4479caa-d0c8-4335-ac65-3dff157d10c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:34.013387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4479caa-d0c8-4335-ac65-3dff157d10c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:09:34.013743 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4479caa-d0c8-4335-ac65-3dff157d10c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:34.015570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4479caa-d0c8-4335-ac65-3dff157d10c3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:34.029213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1095/2213] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:09:34.033657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e774db2-cb99-4287-aabc-e1557b479ca4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:34.045602 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e774db2-cb99-4287-aabc-e1557b479ca4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:34.051255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dfeb4-69dc-498c-9c5a-b28f9f5823f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:34.051860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dfeb4-69dc-498c-9c5a-b28f9f5823f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:34.052264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dfeb4-69dc-498c-9c5a-b28f9f5823f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:34.052541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dfeb4-69dc-498c-9c5a-b28f9f5823f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:34.053182 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dfeb4-69dc-498c-9c5a-b28f9f5823f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:34.055141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dfeb4-69dc-498c-9c5a-b28f9f5823f8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:34.098237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1119/2214] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:09:34 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:34.103710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4479caa-d0c8-4335-ac65-3dff157d10c3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:34.128857 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4479caa-d0c8-4335-ac65-3dff157d10c3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:34.161916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:34.214614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fcddf-fa74-458b-8e36-9b4e202a5afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:34.215304 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fcddf-fa74-458b-8e36-9b4e202a5afb admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:34.215520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fcddf-fa74-458b-8e36-9b4e202a5afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:34.215796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fcddf-fa74-458b-8e36-9b4e202a5afb admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:34.216984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fcddf-fa74-458b-8e36-9b4e202a5afb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:34.218992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fcddf-fa74-458b-8e36-9b4e202a5afb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:34.259000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1096/2215] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:09:34 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:34.261726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dfeb4-69dc-498c-9c5a-b28f9f5823f8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:34.272423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dfeb4-69dc-498c-9c5a-b28f9f5823f8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:34.277491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b13ad-30f6-40e5-b03a-9b5500c8ac6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:34.279006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b13ad-30f6-40e5-b03a-9b5500c8ac6d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:34.279006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b13ad-30f6-40e5-b03a-9b5500c8ac6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:34.279006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b13ad-30f6-40e5-b03a-9b5500c8ac6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:34.279751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b13ad-30f6-40e5-b03a-9b5500c8ac6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:34.281395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b13ad-30f6-40e5-b03a-9b5500c8ac6d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:34.293983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1120/2216] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:09:34.297993 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fcddf-fa74-458b-8e36-9b4e202a5afb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:34.310268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fcddf-fa74-458b-8e36-9b4e202a5afb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:34.315186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb4066-a820-4e53-b009-08b0a5f402e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:34.315876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4066-a820-4e53-b009-08b0a5f402e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:34.316225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4066-a820-4e53-b009-08b0a5f402e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:34.316571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4066-a820-4e53-b009-08b0a5f402e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:34.317275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb4066-a820-4e53-b009-08b0a5f402e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:34.319268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb4066-a820-4e53-b009-08b0a5f402e8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:34.334624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1097/2217] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:34.336193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b13ad-30f6-40e5-b03a-9b5500c8ac6d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:34.352139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b13ad-30f6-40e5-b03a-9b5500c8ac6d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:34.360371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258ac2e-7580-4525-a5f7-c132b0cec339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:34.361281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258ac2e-7580-4525-a5f7-c132b0cec339 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:34.361567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258ac2e-7580-4525-a5f7-c132b0cec339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:34.361978 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258ac2e-7580-4525-a5f7-c132b0cec339 admin admin] PATH_INFO: `/v3/projects/6c71f08e6a5d4be888591f09c3ca3e2d/users/f0062378233848a68c5220abbd7164d3/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:34.392975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258ac2e-7580-4525-a5f7-c132b0cec339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c71f08e6a5d4be888591f09c3ca3e2d, user_id=f0062378233848a68c5220abbd7164d3, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:34.397836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258ac2e-7580-4525-a5f7-c132b0cec339 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:34.427064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1121/2218] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:34 2022] PUT /identity/v3/projects/6c71f08e6a5d4be888591f09c3ca3e2d/users/f0062378233848a68c5220abbd7164d3/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:34.438754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8a73e-98a7-49ff-a4b2-bf53323b6675 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:34.439061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8a73e-98a7-49ff-a4b2-bf53323b6675 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:34.439335 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8a73e-98a7-49ff-a4b2-bf53323b6675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:34.467248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-01f8a73e-98a7-49ff-a4b2-bf53323b6675 None None] MFA Rules not processed for user `f0062378233848a68c5220abbd7164d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:34.468458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-01f8a73e-98a7-49ff-a4b2-bf53323b6675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:34.535310 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1098/2219] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:34.546106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258ac2e-7580-4525-a5f7-c132b0cec339 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:34.547827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2258ac2e-7580-4525-a5f7-c132b0cec339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:34.605436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258ac2e-7580-4525-a5f7-c132b0cec339 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:34.609137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73568d0-c5cf-49f2-9750-e8fd494eae33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:34.609788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73568d0-c5cf-49f2-9750-e8fd494eae33 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:34.610067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73568d0-c5cf-49f2-9750-e8fd494eae33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:34.610331 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73568d0-c5cf-49f2-9750-e8fd494eae33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:34.619345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73568d0-c5cf-49f2-9750-e8fd494eae33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:34.620759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73568d0-c5cf-49f2-9750-e8fd494eae33 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:34.629538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1122/2220] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:38.835702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011165f-8cf9-4091-a2eb-761ae18af90b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:38.836529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011165f-8cf9-4091-a2eb-761ae18af90b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:38.836529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011165f-8cf9-4091-a2eb-761ae18af90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:38.856716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e011165f-8cf9-4091-a2eb-761ae18af90b None None] MFA Rules not processed for user `2ad9fb3dd77d47c190e14cf504ea4d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:38.857968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e011165f-8cf9-4091-a2eb-761ae18af90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:38.919849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1099/2221] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:38.964218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73568d0-c5cf-49f2-9750-e8fd494eae33 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:39.041957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:39.047601 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73568d0-c5cf-49f2-9750-e8fd494eae33 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:39.054661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:39.055593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:39.056065 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:39.056434 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] PATH_INFO: `/v3/users/f0062378233848a68c5220abbd7164d3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:39.057829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0062378233848a68c5220abbd7164d3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:39.059940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:39.123055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0062378233848a68c5220abbd7164d3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:39.140046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0062378233848a68c5220abbd7164d3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:39.147569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0062378233848a68c5220abbd7164d3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:39.156734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0062378233848a68c5220abbd7164d3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:39.157791 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0062378233848a68c5220abbd7164d3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:39.167481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1123/2222] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:09:38 2022] DELETE /identity/v3/users/f0062378233848a68c5220abbd7164d3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:39.172885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b69ff7-d017-42aa-84ad-a49c55c6b30d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:39.173351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b69ff7-d017-42aa-84ad-a49c55c6b30d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:39.173851 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b69ff7-d017-42aa-84ad-a49c55c6b30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:39.195488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f8b69ff7-d017-42aa-84ad-a49c55c6b30d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:39.196873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f8b69ff7-d017-42aa-84ad-a49c55c6b30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:39.246049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:39.275421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1100/2223] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:39.285515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:39.286975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:39.350831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74139b-cf72-4b05-9b45-588c15abaab4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:39.355282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba535370-7393-443d-b392-367eab6f9355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:39.355892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba535370-7393-443d-b392-367eab6f9355 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:39.356174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba535370-7393-443d-b392-367eab6f9355 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:39.356448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba535370-7393-443d-b392-367eab6f9355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:39.366047 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba535370-7393-443d-b392-367eab6f9355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:39.367511 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba535370-7393-443d-b392-367eab6f9355 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:39.377342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1124/2224] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:39.829177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b69ff7-d017-42aa-84ad-a49c55c6b30d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:39.831068 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f8b69ff7-d017-42aa-84ad-a49c55c6b30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:39.851818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b69ff7-d017-42aa-84ad-a49c55c6b30d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:39.859349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:39.860018 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:39.860357 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:39.860633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] PATH_INFO: `/v3/projects/6c71f08e6a5d4be888591f09c3ca3e2d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:39.861877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c71f08e6a5d4be888591f09c3ca3e2d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:39.863810 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:39.936946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c71f08e6a5d4be888591f09c3ca3e2d', 'request_id': 'req-7bce98bf-cf11-442c-bda9-7634e13082f6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:39.943274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c71f08e6a5d4be888591f09c3ca3e2d', 'request_id': 'req-7bce98bf-cf11-442c-bda9-7634e13082f6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:39.950355 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c71f08e6a5d4be888591f09c3ca3e2d', 'request_id': 'req-7bce98bf-cf11-442c-bda9-7634e13082f6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:39.965142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] The token cache is being invalidate because project 6c71f08e6a5d4be888591f09c3ca3e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:09:39.965620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:39.966256 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7bce98bf-cf11-442c-bda9-7634e13082f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:09:39.967905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1101/2225] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:09:39 2022] DELETE /identity/v3/projects/6c71f08e6a5d4be888591f09c3ca3e2d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:39.975250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01459eb-1e3b-4861-ac05-ef0bfd14666f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:39.975765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01459eb-1e3b-4861-ac05-ef0bfd14666f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:39.975889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01459eb-1e3b-4861-ac05-ef0bfd14666f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:39.999193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d01459eb-1e3b-4861-ac05-ef0bfd14666f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:40.000779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d01459eb-1e3b-4861-ac05-ef0bfd14666f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:40.062029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1125/2226] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:40.069939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fb296-dfb9-4327-ad44-58a1fd85184a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:40.070226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fb296-dfb9-4327-ad44-58a1fd85184a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:40.070503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fb296-dfb9-4327-ad44-58a1fd85184a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:40.090877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f69fb296-dfb9-4327-ad44-58a1fd85184a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:40.092103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f69fb296-dfb9-4327-ad44-58a1fd85184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:40.099862 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1102/2227] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:09:40.104075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01459eb-1e3b-4861-ac05-ef0bfd14666f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:40.113730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01459eb-1e3b-4861-ac05-ef0bfd14666f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:40.118819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adab269-91d8-46b9-a1d5-bad542cbd1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:40.119444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adab269-91d8-46b9-a1d5-bad542cbd1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:40.119724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adab269-91d8-46b9-a1d5-bad542cbd1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:40.119993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adab269-91d8-46b9-a1d5-bad542cbd1d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:40.120632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adab269-91d8-46b9-a1d5-bad542cbd1d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:09:40.120932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adab269-91d8-46b9-a1d5-bad542cbd1d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:40.122512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adab269-91d8-46b9-a1d5-bad542cbd1d4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:40.134819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1126/2228] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:09:40.139397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fb296-dfb9-4327-ad44-58a1fd85184a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:40.148850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fb296-dfb9-4327-ad44-58a1fd85184a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:40.154245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d36d3-b49a-4ce3-a0df-54e63abd41c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:40.154848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d36d3-b49a-4ce3-a0df-54e63abd41c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:40.155275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d36d3-b49a-4ce3-a0df-54e63abd41c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:40.155578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d36d3-b49a-4ce3-a0df-54e63abd41c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:40.156273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d36d3-b49a-4ce3-a0df-54e63abd41c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:40.158376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d36d3-b49a-4ce3-a0df-54e63abd41c3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:40.194304 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1103/2229] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:09:40 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:09:40.200237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adab269-91d8-46b9-a1d5-bad542cbd1d4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:40.213172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adab269-91d8-46b9-a1d5-bad542cbd1d4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:40.217213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:40.285327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe188a-5322-425c-a5e3-339f4394f861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:40.285939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe188a-5322-425c-a5e3-339f4394f861 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:40.286239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe188a-5322-425c-a5e3-339f4394f861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:40.286503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe188a-5322-425c-a5e3-339f4394f861 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:40.287177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe188a-5322-425c-a5e3-339f4394f861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:40.289098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe188a-5322-425c-a5e3-339f4394f861 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:40.311488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1127/2230] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:09:40 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:09:40.315347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d36d3-b49a-4ce3-a0df-54e63abd41c3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:40.326638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d36d3-b49a-4ce3-a0df-54e63abd41c3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:40.331690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7115eec-564e-482e-a259-8f329753521d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:40.332294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7115eec-564e-482e-a259-8f329753521d admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:40.332572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7115eec-564e-482e-a259-8f329753521d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:40.332845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7115eec-564e-482e-a259-8f329753521d admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:40.333484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7115eec-564e-482e-a259-8f329753521d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:40.335124 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7115eec-564e-482e-a259-8f329753521d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:40.345918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1104/2231] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:40.350179 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe188a-5322-425c-a5e3-339f4394f861 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:40.360743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe188a-5322-425c-a5e3-339f4394f861 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:40.365483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cb085-c54a-484d-b370-a7838060526e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:40.366084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cb085-c54a-484d-b370-a7838060526e admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:40.366367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cb085-c54a-484d-b370-a7838060526e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:40.366641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cb085-c54a-484d-b370-a7838060526e admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:40.367325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cb085-c54a-484d-b370-a7838060526e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:40.368947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cb085-c54a-484d-b370-a7838060526e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:40.381738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1128/2232] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:09:40.386451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7115eec-564e-482e-a259-8f329753521d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:40.396201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7115eec-564e-482e-a259-8f329753521d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:40.401580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78011f-63d9-4902-808e-c5296f4eb4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:40.402238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78011f-63d9-4902-808e-c5296f4eb4d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:40.402497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78011f-63d9-4902-808e-c5296f4eb4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:40.402764 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78011f-63d9-4902-808e-c5296f4eb4d3 admin admin] PATH_INFO: `/v3/projects/83a7aefc916343efb12f3b52449cd175/users/7bd93376223d4fd1aac25e6b12bf370a/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:40.417016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78011f-63d9-4902-808e-c5296f4eb4d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a7aefc916343efb12f3b52449cd175, user_id=7bd93376223d4fd1aac25e6b12bf370a, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:40.419683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78011f-63d9-4902-808e-c5296f4eb4d3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:40.432639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1105/2233] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:09:40 2022] PUT /identity/v3/projects/83a7aefc916343efb12f3b52449cd175/users/7bd93376223d4fd1aac25e6b12bf370a/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:09:40.444022 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcee037-6940-4914-83e6-9d6815c6fd3c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:40.444485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcee037-6940-4914-83e6-9d6815c6fd3c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:40.444720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcee037-6940-4914-83e6-9d6815c6fd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:40.473462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-fbcee037-6940-4914-83e6-9d6815c6fd3c None None] MFA Rules not processed for user `7bd93376223d4fd1aac25e6b12bf370a`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:40.474930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fbcee037-6940-4914-83e6-9d6815c6fd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:40.540679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1129/2234] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:09:40.551728 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78011f-63d9-4902-808e-c5296f4eb4d3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:09:40.553656 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6e78011f-63d9-4902-808e-c5296f4eb4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:40.611077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78011f-63d9-4902-808e-c5296f4eb4d3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:09:40.614723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21a21e-d830-4096-87eb-dfbc417995a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:09:40.615349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21a21e-d830-4096-87eb-dfbc417995a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:40.615625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21a21e-d830-4096-87eb-dfbc417995a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:40.615891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21a21e-d830-4096-87eb-dfbc417995a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:40.624780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21a21e-d830-4096-87eb-dfbc417995a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:09:40.626792 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21a21e-d830-4096-87eb-dfbc417995a9 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:09:40.636338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1106/2235] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:09:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:09:50.537800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:50.631610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:09:59.736315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b63d6-50d4-4639-a420-d723cf6becf4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:09:59.736849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b63d6-50d4-4639-a420-d723cf6becf4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:09:59.736849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b63d6-50d4-4639-a420-d723cf6becf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:09:59.756116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ed8b63d6-50d4-4639-a420-d723cf6becf4 None None] MFA Rules not processed for user `bc8c2a5413c943ed94d76eceb2720f17`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:09:59.757350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ed8b63d6-50d4-4639-a420-d723cf6becf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:09:59.828407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1130/2236] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:09:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:10:09.820752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=25 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:10:16.133142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81afbf-546c-4ec3-9703-e7cab4d78af8 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:10:16.133643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81afbf-546c-4ec3-9703-e7cab4d78af8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:10:16.133747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81afbf-546c-4ec3-9703-e7cab4d78af8 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:10:16.135115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1107/2237] 10.222.0.16 () {54 vars in 961 bytes} [Thu Sep 22 17:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:10:16.140272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b9efc-3d3c-47cf-946f-e987af25315d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:10:16.140809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b9efc-3d3c-47cf-946f-e987af25315d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:10:16.140936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b9efc-3d3c-47cf-946f-e987af25315d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:10:16.161970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-045b9efc-3d3c-47cf-946f-e987af25315d None None] MFA Rules not processed for user `81088a327fd448ad8fec555f3d914be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:10:16.163258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-045b9efc-3d3c-47cf-946f-e987af25315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:10:16.235616 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1131/2238] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Sep 22 17:10:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:10:16.243176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81afbf-546c-4ec3-9703-e7cab4d78af8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:10:16.256842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81afbf-546c-4ec3-9703-e7cab4d78af8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:10:16.261923 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3592c9-6cee-4bf3-9e51-5c14b9d23f64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:10:16.262755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3592c9-6cee-4bf3-9e51-5c14b9d23f64 service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:10:16.262850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3592c9-6cee-4bf3-9e51-5c14b9d23f64 service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:10:16.263090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3592c9-6cee-4bf3-9e51-5c14b9d23f64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:10:16.264928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5a3592c9-6cee-4bf3-9e51-5c14b9d23f64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:10:16.268051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b9efc-3d3c-47cf-946f-e987af25315d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:10:16.269485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-045b9efc-3d3c-47cf-946f-e987af25315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:10:16.278605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3592c9-6cee-4bf3-9e51-5c14b9d23f64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:10:16.280400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3592c9-6cee-4bf3-9e51-5c14b9d23f64 service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:10:16.284954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b9efc-3d3c-47cf-946f-e987af25315d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:10:16.290995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4f5d4-a91b-47a6-95a2-4b135f1b852a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:10:16.291162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1108/2239] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:10:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:10:16.292558 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f5d4-a91b-47a6-95a2-4b135f1b852a service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:10:16.292842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f5d4-a91b-47a6-95a2-4b135f1b852a service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:10:16.293205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f5d4-a91b-47a6-95a2-4b135f1b852a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:10:16.304787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4f5d4-a91b-47a6-95a2-4b135f1b852a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:10:16.306369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4f5d4-a91b-47a6-95a2-4b135f1b852a service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:10:16.317660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1132/2240] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:10:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:10:25.998178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3592c9-6cee-4bf3-9e51-5c14b9d23f64 service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:10:26.081188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3592c9-6cee-4bf3-9e51-5c14b9d23f64 service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:10:26.085791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:10:26.086423 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:10:26.086706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:10:26.087056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] PATH_INFO: `/v3/users/7bd93376223d4fd1aac25e6b12bf370a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:10:26.088841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd93376223d4fd1aac25e6b12bf370a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:10:26.090769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:10:26.141110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd93376223d4fd1aac25e6b12bf370a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:10:26.154468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd93376223d4fd1aac25e6b12bf370a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:10:26.167921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd93376223d4fd1aac25e6b12bf370a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:10:26.168728 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd93376223d4fd1aac25e6b12bf370a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:10:26.181447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd93376223d4fd1aac25e6b12bf370a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:10:26.193583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1109/2241] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:10:25 2022] DELETE /identity/v3/users/7bd93376223d4fd1aac25e6b12bf370a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:10:26.198730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadceed4-ecf9-4a90-ad7e-c2f4e5b75427 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:10:26.199354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadceed4-ecf9-4a90-ad7e-c2f4e5b75427 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:10:26.199354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadceed4-ecf9-4a90-ad7e-c2f4e5b75427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:10:26.226236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-cadceed4-ecf9-4a90-ad7e-c2f4e5b75427 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:10:26.227975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cadceed4-ecf9-4a90-ad7e-c2f4e5b75427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:10:26.311320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1133/2242] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:10:26.318568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:10:26.320075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:10:26.394348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7aaa76-c22d-4a1d-87ea-406e63254d8b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:10:26.403281 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039edc9-cbe7-40bf-86d0-d84615cdb273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:10:26.405157 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039edc9-cbe7-40bf-86d0-d84615cdb273 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:10:26.405480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039edc9-cbe7-40bf-86d0-d84615cdb273 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:10:26.405900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039edc9-cbe7-40bf-86d0-d84615cdb273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:10:26.421528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039edc9-cbe7-40bf-86d0-d84615cdb273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:10:26.425087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039edc9-cbe7-40bf-86d0-d84615cdb273 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:10:26.444129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1110/2243] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:10:26.886606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadceed4-ecf9-4a90-ad7e-c2f4e5b75427 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:10:26.888693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cadceed4-ecf9-4a90-ad7e-c2f4e5b75427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:10:26.903649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadceed4-ecf9-4a90-ad7e-c2f4e5b75427 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:10:26.908347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:10:26.909171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:10:26.909505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:10:26.909931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] PATH_INFO: `/v3/projects/83a7aefc916343efb12f3b52449cd175` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:10:26.911084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=83a7aefc916343efb12f3b52449cd175)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:10:26.914086 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:10:26.995575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83a7aefc916343efb12f3b52449cd175', 'request_id': 'req-a5918d44-aae6-41f8-8786-4f0ac876a2bf'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:10:27.006690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83a7aefc916343efb12f3b52449cd175', 'request_id': 'req-a5918d44-aae6-41f8-8786-4f0ac876a2bf'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:10:27.015027 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83a7aefc916343efb12f3b52449cd175', 'request_id': 'req-a5918d44-aae6-41f8-8786-4f0ac876a2bf'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:10:27.033028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] The token cache is being invalidate because project 83a7aefc916343efb12f3b52449cd175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:10:27.033326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:10:27.034333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:10:27.036325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1134/2244] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:10:26 2022] DELETE /identity/v3/projects/83a7aefc916343efb12f3b52449cd175 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:10:36.432413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:10:37.028534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:10:51.181762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da21f2c-bd57-4688-aad0-e4bb786b2eb2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:10:51.182582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da21f2c-bd57-4688-aad0-e4bb786b2eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:10:51.182582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da21f2c-bd57-4688-aad0-e4bb786b2eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:10:51.214899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8da21f2c-bd57-4688-aad0-e4bb786b2eb2 None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:10:51.217185 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8da21f2c-bd57-4688-aad0-e4bb786b2eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:10:51.332456 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1111/2245] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:10:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:10:51.400881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:10:51.402506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:10:51.483580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5918d44-aae6-41f8-8786-4f0ac876a2bf admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:10:51.490142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c5a71-1452-473e-8966-2d06a9561dfb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:10:51.490978 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c5a71-1452-473e-8966-2d06a9561dfb service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:10:51.491458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c5a71-1452-473e-8966-2d06a9561dfb service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:10:51.491874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c5a71-1452-473e-8966-2d06a9561dfb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:10:51.503891 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c5a71-1452-473e-8966-2d06a9561dfb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:10:51.505961 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c5a71-1452-473e-8966-2d06a9561dfb service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:10:51.516063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1135/2246] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:10:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:10:59.028079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d1794-dbc7-49ad-bf20-984e92cf1810 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:10:59.029024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d1794-dbc7-49ad-bf20-984e92cf1810 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:10:59.029024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d1794-dbc7-49ad-bf20-984e92cf1810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:10:59.072815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-116d1794-dbc7-49ad-bf20-984e92cf1810 None None] MFA Rules not processed for user `4386cce77ec047f480e9999aeb103da5`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:10:59.075631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-116d1794-dbc7-49ad-bf20-984e92cf1810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:10:59.188584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1112/2247] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:10:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:11:01.512222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=32 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:11:02.163101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c5a71-1452-473e-8966-2d06a9561dfb service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:02.164653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5c5c5a71-1452-473e-8966-2d06a9561dfb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:02.333555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c5a71-1452-473e-8966-2d06a9561dfb service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:02.339690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d0d99-7a35-430f-a1be-eb9d57863faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:02.340780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d0d99-7a35-430f-a1be-eb9d57863faa service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:02.341238 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d0d99-7a35-430f-a1be-eb9d57863faa service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:02.341694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d0d99-7a35-430f-a1be-eb9d57863faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:02.344256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-742d0d99-7a35-430f-a1be-eb9d57863faa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:02.475791 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d0d99-7a35-430f-a1be-eb9d57863faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:02.477584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d0d99-7a35-430f-a1be-eb9d57863faa service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:02.499259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1136/2248] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:11:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:11:09.175061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=50 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:11:11.792203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b48f4-3c7c-4b6d-9e71-2ed2a17c2f98 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:11.793953 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b48f4-3c7c-4b6d-9e71-2ed2a17c2f98 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:11.794670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b48f4-3c7c-4b6d-9e71-2ed2a17c2f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:11.831233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-490b48f4-3c7c-4b6d-9e71-2ed2a17c2f98 None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:11:11.834375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-490b48f4-3c7c-4b6d-9e71-2ed2a17c2f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:11.847929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1113/2249] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 22 17:11:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:11:11.865525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d0d99-7a35-430f-a1be-eb9d57863faa service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:11.887431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d0d99-7a35-430f-a1be-eb9d57863faa service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:11.895691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb091ce4-9d24-47ec-8a46-c005ab820c70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:11.897007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb091ce4-9d24-47ec-8a46-c005ab820c70 service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:11.897545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb091ce4-9d24-47ec-8a46-c005ab820c70 service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:11.897819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb091ce4-9d24-47ec-8a46-c005ab820c70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:11.909856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb091ce4-9d24-47ec-8a46-c005ab820c70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:11.911615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb091ce4-9d24-47ec-8a46-c005ab820c70 service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:11.921304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1137/2250] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:11:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:11:14.098177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b48f4-3c7c-4b6d-9e71-2ed2a17c2f98 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:14.101163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-490b48f4-3c7c-4b6d-9e71-2ed2a17c2f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:14.188859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b48f4-3c7c-4b6d-9e71-2ed2a17c2f98 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:14.207625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c93a12-6e77-4abc-af00-1233f7abc8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:14.208166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c93a12-6e77-4abc-af00-1233f7abc8da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:14.208441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c93a12-6e77-4abc-af00-1233f7abc8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:14.208731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c93a12-6e77-4abc-af00-1233f7abc8da admin admin] PATH_INFO: `/v3/users/2ad9fb3dd77d47c190e14cf504ea4d3d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:14.210424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c93a12-6e77-4abc-af00-1233f7abc8da admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ad9fb3dd77d47c190e14cf504ea4d3d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:14.213446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c93a12-6e77-4abc-af00-1233f7abc8da admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:14.294670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26c93a12-6e77-4abc-af00-1233f7abc8da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ad9fb3dd77d47c190e14cf504ea4d3d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:11:14.314543 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26c93a12-6e77-4abc-af00-1233f7abc8da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ad9fb3dd77d47c190e14cf504ea4d3d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:11:14.342737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26c93a12-6e77-4abc-af00-1233f7abc8da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ad9fb3dd77d47c190e14cf504ea4d3d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:11:14.344246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26c93a12-6e77-4abc-af00-1233f7abc8da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ad9fb3dd77d47c190e14cf504ea4d3d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:11:14.360907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26c93a12-6e77-4abc-af00-1233f7abc8da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ad9fb3dd77d47c190e14cf504ea4d3d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:11:14.372441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1114/2251] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:11:14 2022] DELETE /identity/v3/users/2ad9fb3dd77d47c190e14cf504ea4d3d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:11:14.568981 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb091ce4-9d24-47ec-8a46-c005ab820c70 service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:14.583849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb091ce4-9d24-47ec-8a46-c005ab820c70 service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:14.651974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:14.652984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:14.654359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:14.654884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] PATH_INFO: `/v3/projects/6c516bf5715a42cd860981f82fc58b15` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:14.657180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c516bf5715a42cd860981f82fc58b15)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:14.659964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:14.713776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:11:14.726527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c516bf5715a42cd860981f82fc58b15', 'request_id': 'req-6f81faac-f7f7-48fe-9422-a3d45a331d9c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:11:14.736847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c516bf5715a42cd860981f82fc58b15', 'request_id': 'req-6f81faac-f7f7-48fe-9422-a3d45a331d9c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:11:14.746138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c516bf5715a42cd860981f82fc58b15', 'request_id': 'req-6f81faac-f7f7-48fe-9422-a3d45a331d9c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:11:14.763508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] The token cache is being invalidate because project 6c516bf5715a42cd860981f82fc58b15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:11:14.763951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:11:14.764773 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6f81faac-f7f7-48fe-9422-a3d45a331d9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:11:14.766654 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1138/2252] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:11:14 2022] DELETE /identity/v3/projects/6c516bf5715a42cd860981f82fc58b15 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:11:14.771292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced45eb9-14aa-4a7f-aa5e-13169fa89a33 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:14.771606 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced45eb9-14aa-4a7f-aa5e-13169fa89a33 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:14.771884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced45eb9-14aa-4a7f-aa5e-13169fa89a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:14.793525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ced45eb9-14aa-4a7f-aa5e-13169fa89a33 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:11:14.795062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ced45eb9-14aa-4a7f-aa5e-13169fa89a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:14.856618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1115/2253] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:11:14.864253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328c0b3-4014-4e38-82ab-2a83e1dfb1f6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:14.864536 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328c0b3-4014-4e38-82ab-2a83e1dfb1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:14.864810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328c0b3-4014-4e38-82ab-2a83e1dfb1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:14.884844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5328c0b3-4014-4e38-82ab-2a83e1dfb1f6 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:11:14.886031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5328c0b3-4014-4e38-82ab-2a83e1dfb1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:14.894210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1139/2254] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:11:14.898506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced45eb9-14aa-4a7f-aa5e-13169fa89a33 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:14.908954 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced45eb9-14aa-4a7f-aa5e-13169fa89a33 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:14.913782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921bf96-05c0-4147-8bc0-0c914bfe8102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:14.914391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921bf96-05c0-4147-8bc0-0c914bfe8102 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:14.914671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921bf96-05c0-4147-8bc0-0c914bfe8102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:14.914994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921bf96-05c0-4147-8bc0-0c914bfe8102 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:14.916208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921bf96-05c0-4147-8bc0-0c914bfe8102 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:11:14.916537 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921bf96-05c0-4147-8bc0-0c914bfe8102 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:14.918145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921bf96-05c0-4147-8bc0-0c914bfe8102 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:14.927514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:11:14.931446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1116/2255] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 17:11:14.935645 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328c0b3-4014-4e38-82ab-2a83e1dfb1f6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:14.945047 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328c0b3-4014-4e38-82ab-2a83e1dfb1f6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:14.950383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac62ef-bd46-4649-83ee-c8e1a23e425f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:14.951024 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac62ef-bd46-4649-83ee-c8e1a23e425f admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:14.951332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac62ef-bd46-4649-83ee-c8e1a23e425f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:14.951604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac62ef-bd46-4649-83ee-c8e1a23e425f admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:14.952247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac62ef-bd46-4649-83ee-c8e1a23e425f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:14.954137 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac62ef-bd46-4649-83ee-c8e1a23e425f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:14.987454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1140/2256] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:11:14 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:11:14.991741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921bf96-05c0-4147-8bc0-0c914bfe8102 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:15.001966 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921bf96-05c0-4147-8bc0-0c914bfe8102 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:15.069497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32e428-b50b-4b36-8867-6a0607b185d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:15.070560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e428-b50b-4b36-8867-6a0607b185d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:15.071065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e428-b50b-4b36-8867-6a0607b185d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:15.072071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e428-b50b-4b36-8867-6a0607b185d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:15.073261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32e428-b50b-4b36-8867-6a0607b185d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:15.076588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32e428-b50b-4b36-8867-6a0607b185d1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:15.101250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1117/2257] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:11:14 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:11:15.105521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac62ef-bd46-4649-83ee-c8e1a23e425f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:15.116529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac62ef-bd46-4649-83ee-c8e1a23e425f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:15.121606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392376b-7385-4fd3-9abb-e2db095c76c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:15.122254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392376b-7385-4fd3-9abb-e2db095c76c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:15.122643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392376b-7385-4fd3-9abb-e2db095c76c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:15.122914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392376b-7385-4fd3-9abb-e2db095c76c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:15.123686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392376b-7385-4fd3-9abb-e2db095c76c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:15.125405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392376b-7385-4fd3-9abb-e2db095c76c5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:15.137096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1141/2258] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:11:15.141231 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32e428-b50b-4b36-8867-6a0607b185d1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:15.151947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32e428-b50b-4b36-8867-6a0607b185d1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:15.156688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578c551-0f16-445e-b974-56bc17db5d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:15.157284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578c551-0f16-445e-b974-56bc17db5d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:15.157560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578c551-0f16-445e-b974-56bc17db5d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:15.157831 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578c551-0f16-445e-b974-56bc17db5d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:15.158464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578c551-0f16-445e-b974-56bc17db5d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:15.160079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578c551-0f16-445e-b974-56bc17db5d39 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:15.170996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1118/2259] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:11:15.175662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392376b-7385-4fd3-9abb-e2db095c76c5 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:15.185870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392376b-7385-4fd3-9abb-e2db095c76c5 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:15.190550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b691f6-6b2c-475b-b1d2-292ee1a5894f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:15.191229 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b691f6-6b2c-475b-b1d2-292ee1a5894f admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:15.191504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b691f6-6b2c-475b-b1d2-292ee1a5894f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:15.191774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b691f6-6b2c-475b-b1d2-292ee1a5894f admin admin] PATH_INFO: `/v3/projects/5fca6eabbca24bc09aee7e259ca82562/users/c2060b33d0a24e01a96ab3fb02b4b045/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:15.207507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b691f6-6b2c-475b-b1d2-292ee1a5894f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fca6eabbca24bc09aee7e259ca82562, user_id=c2060b33d0a24e01a96ab3fb02b4b045, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:15.210205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b691f6-6b2c-475b-b1d2-292ee1a5894f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:15.224499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1142/2260] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:11:15 2022] PUT /identity/v3/projects/5fca6eabbca24bc09aee7e259ca82562/users/c2060b33d0a24e01a96ab3fb02b4b045/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:11:15.229610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d617f-6691-4515-bb2c-32220db6c72c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:15.229976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d617f-6691-4515-bb2c-32220db6c72c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:15.230246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d617f-6691-4515-bb2c-32220db6c72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:15.249014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7a9d617f-6691-4515-bb2c-32220db6c72c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:11:15.250114 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7a9d617f-6691-4515-bb2c-32220db6c72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:15.308430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1119/2261] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:11:15.318121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b691f6-6b2c-475b-b1d2-292ee1a5894f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:15.319841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-45b691f6-6b2c-475b-b1d2-292ee1a5894f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:15.376814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b691f6-6b2c-475b-b1d2-292ee1a5894f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:15.380511 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa5330-78b3-4353-bd08-1f1b2e7dbd2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:15.381115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa5330-78b3-4353-bd08-1f1b2e7dbd2b service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:15.381398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa5330-78b3-4353-bd08-1f1b2e7dbd2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:15.381673 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa5330-78b3-4353-bd08-1f1b2e7dbd2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:15.391842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa5330-78b3-4353-bd08-1f1b2e7dbd2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:15.393033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa5330-78b3-4353-bd08-1f1b2e7dbd2b service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:15.403968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1143/2262] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:11:20.320704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb76495-0897-432f-9d0f-0eac27977c42 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:20.321917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb76495-0897-432f-9d0f-0eac27977c42 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:20.321917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb76495-0897-432f-9d0f-0eac27977c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:20.359532 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-deb76495-0897-432f-9d0f-0eac27977c42 None None] MFA Rules not processed for user `c2060b33d0a24e01a96ab3fb02b4b045`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:11:20.361237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-deb76495-0897-432f-9d0f-0eac27977c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:20.435916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1120/2263] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:11:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:11:20.440458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa5330-78b3-4353-bd08-1f1b2e7dbd2b service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:20.451202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa5330-78b3-4353-bd08-1f1b2e7dbd2b service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:20.455912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c13e4-80ab-4c79-ba6b-410d5d3792af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:20.456512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c13e4-80ab-4c79-ba6b-410d5d3792af admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:20.456818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c13e4-80ab-4c79-ba6b-410d5d3792af admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:20.457091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c13e4-80ab-4c79-ba6b-410d5d3792af admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:20.457727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c13e4-80ab-4c79-ba6b-410d5d3792af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:20.459675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c13e4-80ab-4c79-ba6b-410d5d3792af admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:20.496441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1144/2264] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:11:20 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:11:20.500619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb76495-0897-432f-9d0f-0eac27977c42 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:20.516202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb76495-0897-432f-9d0f-0eac27977c42 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:20.525261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:11:20.660975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed158972-6faf-460b-b9d9-4c9870f3ba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:20.661744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed158972-6faf-460b-b9d9-4c9870f3ba4e admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:20.662089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed158972-6faf-460b-b9d9-4c9870f3ba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:20.662483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed158972-6faf-460b-b9d9-4c9870f3ba4e admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:20.663286 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed158972-6faf-460b-b9d9-4c9870f3ba4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:20.665403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed158972-6faf-460b-b9d9-4c9870f3ba4e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:20.704469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1121/2265] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:11:20 2022] POST /identity/v3/users => generated 296 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:11:20.706285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c13e4-80ab-4c79-ba6b-410d5d3792af admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:20.743860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c13e4-80ab-4c79-ba6b-410d5d3792af admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:20.753794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3ae75-df4d-42d2-a8ca-9e75c5be4036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:20.755217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3ae75-df4d-42d2-a8ca-9e75c5be4036 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:20.756094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3ae75-df4d-42d2-a8ca-9e75c5be4036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:20.757109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3ae75-df4d-42d2-a8ca-9e75c5be4036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:20.761662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3ae75-df4d-42d2-a8ca-9e75c5be4036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:20.769260 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3ae75-df4d-42d2-a8ca-9e75c5be4036 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:20.795899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1145/2266] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:11:20.801695 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed158972-6faf-460b-b9d9-4c9870f3ba4e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:20.819975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed158972-6faf-460b-b9d9-4c9870f3ba4e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:20.828706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7474917-b714-4d43-a4b3-5124d4822a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:20.829851 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7474917-b714-4d43-a4b3-5124d4822a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:20.830346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7474917-b714-4d43-a4b3-5124d4822a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:20.830836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7474917-b714-4d43-a4b3-5124d4822a33 admin admin] PATH_INFO: `/v3/projects/ec3eb12cabb04fd8bba960c1432f65e1/users/da9803645097465887035bf5fe7de010/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:20.861721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7474917-b714-4d43-a4b3-5124d4822a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec3eb12cabb04fd8bba960c1432f65e1, user_id=da9803645097465887035bf5fe7de010, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:20.867054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7474917-b714-4d43-a4b3-5124d4822a33 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:20.893704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1122/2267] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:11:20 2022] PUT /identity/v3/projects/ec3eb12cabb04fd8bba960c1432f65e1/users/da9803645097465887035bf5fe7de010/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:11:20.899855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3ae75-df4d-42d2-a8ca-9e75c5be4036 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:20.917794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3ae75-df4d-42d2-a8ca-9e75c5be4036 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:20.971726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:11:21.032071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc550be-9437-4001-a971-feb70878b9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:21.033305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc550be-9437-4001-a971-feb70878b9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:21.033305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc550be-9437-4001-a971-feb70878b9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:21.035347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc550be-9437-4001-a971-feb70878b9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:21.036061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc550be-9437-4001-a971-feb70878b9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:21.037746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc550be-9437-4001-a971-feb70878b9b8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:21.053302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7474917-b714-4d43-a4b3-5124d4822a33 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:21.054032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1146/2268] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:11:21.066499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7474917-b714-4d43-a4b3-5124d4822a33 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:21.075493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a3eab-33f0-4bfd-a0a1-53289d954df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:21.076297 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a3eab-33f0-4bfd-a0a1-53289d954df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:21.076609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a3eab-33f0-4bfd-a0a1-53289d954df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:21.076891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a3eab-33f0-4bfd-a0a1-53289d954df3 admin admin] PATH_INFO: `/v3/projects/ec3eb12cabb04fd8bba960c1432f65e1/users/da9803645097465887035bf5fe7de010/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:21.079860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a3eab-33f0-4bfd-a0a1-53289d954df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec3eb12cabb04fd8bba960c1432f65e1, user_id=da9803645097465887035bf5fe7de010, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:21.082343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a3eab-33f0-4bfd-a0a1-53289d954df3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:21.095798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1123/2269] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:11:21 2022] PUT /identity/v3/projects/ec3eb12cabb04fd8bba960c1432f65e1/users/da9803645097465887035bf5fe7de010/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:11:27.277284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8058d9-125a-4721-9e49-f376c647b6d3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:27.278256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8058d9-125a-4721-9e49-f376c647b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:27.278256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8058d9-125a-4721-9e49-f376c647b6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:27.325762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8e8058d9-125a-4721-9e49-f376c647b6d3 None None] MFA Rules not processed for user `da9803645097465887035bf5fe7de010`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:11:27.328010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8e8058d9-125a-4721-9e49-f376c647b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:27.447899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1147/2270] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:11:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:11:27.468683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a3eab-33f0-4bfd-a0a1-53289d954df3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:27.470690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e79a3eab-33f0-4bfd-a0a1-53289d954df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:27.592121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a3eab-33f0-4bfd-a0a1-53289d954df3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:27.602332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aae9d6-fa2b-43fe-87fd-6720012dcae7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:27.603432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aae9d6-fa2b-43fe-87fd-6720012dcae7 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:27.603937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aae9d6-fa2b-43fe-87fd-6720012dcae7 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:27.604416 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aae9d6-fa2b-43fe-87fd-6720012dcae7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:27.740170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aae9d6-fa2b-43fe-87fd-6720012dcae7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:27.744012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aae9d6-fa2b-43fe-87fd-6720012dcae7 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:27.781833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:11:27.807775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1124/2271] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:11:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:11:37.442901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=48 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:11:37.808245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:11:46.104228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a6e05-70ac-477d-aba3-9886de2706b6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:46.104771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a6e05-70ac-477d-aba3-9886de2706b6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:46.104771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a6e05-70ac-477d-aba3-9886de2706b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:46.124198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-4a7a6e05-70ac-477d-aba3-9886de2706b6 None None] MFA Rules not processed for user `5feea3b3b7e4401ea7db242cb60fdfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:11:46.125531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4a7a6e05-70ac-477d-aba3-9886de2706b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:46.196538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1148/2272] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:11:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:11:52.751065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab873b1-b440-45af-951e-bb469e820f61 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:52.751554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab873b1-b440-45af-951e-bb469e820f61 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:52.751662 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab873b1-b440-45af-951e-bb469e820f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:52.781940 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bab873b1-b440-45af-951e-bb469e820f61 None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:11:52.783101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bab873b1-b440-45af-951e-bb469e820f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:52.871432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1125/2273] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:11:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:11:52.885615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a6e05-70ac-477d-aba3-9886de2706b6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:11:52.887362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4a7a6e05-70ac-477d-aba3-9886de2706b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:11:52.973104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a6e05-70ac-477d-aba3-9886de2706b6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:11:52.978658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedac8b-5f83-4d57-b8b1-5f47d501e8a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:11:52.979555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedac8b-5f83-4d57-b8b1-5f47d501e8a3 service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:11:52.979993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedac8b-5f83-4d57-b8b1-5f47d501e8a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:11:52.980428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedac8b-5f83-4d57-b8b1-5f47d501e8a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:11:52.989946 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aedac8b-5f83-4d57-b8b1-5f47d501e8a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:11:52.992206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aedac8b-5f83-4d57-b8b1-5f47d501e8a3 service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:11:53.005507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1149/2274] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:11:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:12:02.847889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=25 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:12:03.000897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=57 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:12:06.782441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab873b1-b440-45af-951e-bb469e820f61 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:06.844224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab873b1-b440-45af-951e-bb469e820f61 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:06.847887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfecad87-78b1-40e7-83a1-a3a76fe431da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:06.848496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecad87-78b1-40e7-83a1-a3a76fe431da service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:06.848774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecad87-78b1-40e7-83a1-a3a76fe431da service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:06.849048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecad87-78b1-40e7-83a1-a3a76fe431da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:06.851435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cfecad87-78b1-40e7-83a1-a3a76fe431da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:06.866704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfecad87-78b1-40e7-83a1-a3a76fe431da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:06.868175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfecad87-78b1-40e7-83a1-a3a76fe431da service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:06.879572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1126/2275] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:12:07.253220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f8e49-9b30-49c8-b901-e6fb7681875e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:07.253779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f8e49-9b30-49c8-b901-e6fb7681875e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:07.253779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f8e49-9b30-49c8-b901-e6fb7681875e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:07.274464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-715f8e49-9b30-49c8-b901-e6fb7681875e None None] MFA Rules not processed for user `c2060b33d0a24e01a96ab3fb02b4b045`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:07.275813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-715f8e49-9b30-49c8-b901-e6fb7681875e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:07.283272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1150/2276] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:12:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:12:07.322584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfecad87-78b1-40e7-83a1-a3a76fe431da service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:07.324009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cfecad87-78b1-40e7-83a1-a3a76fe431da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:07.384713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfecad87-78b1-40e7-83a1-a3a76fe431da service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:07.389589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dba725-df4a-4d5c-b95f-e7e1ce3fa733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:07.390206 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dba725-df4a-4d5c-b95f-e7e1ce3fa733 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:07.390482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dba725-df4a-4d5c-b95f-e7e1ce3fa733 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:07.390773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dba725-df4a-4d5c-b95f-e7e1ce3fa733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:07.393028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-42dba725-df4a-4d5c-b95f-e7e1ce3fa733 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:07.417853 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dba725-df4a-4d5c-b95f-e7e1ce3fa733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:07.419437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dba725-df4a-4d5c-b95f-e7e1ce3fa733 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:07.431660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1127/2277] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:12:17.267566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:12:17.426285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=52 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:12:17.678488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d10bff-1e26-4ee5-85ed-479c7904353a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:17.679172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d10bff-1e26-4ee5-85ed-479c7904353a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:17.679623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d10bff-1e26-4ee5-85ed-479c7904353a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:17.684183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1151/2278] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 17:12:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:12:17.694448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab50f6-b021-45ac-af9f-1f10971ca563 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:17.694734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab50f6-b021-45ac-af9f-1f10971ca563 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:17.695031 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab50f6-b021-45ac-af9f-1f10971ca563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:17.727133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f4ab50f6-b021-45ac-af9f-1f10971ca563 None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:17.729286 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f4ab50f6-b021-45ac-af9f-1f10971ca563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:17.748454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1128/2279] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 17:12:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:12:17.764237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d10bff-1e26-4ee5-85ed-479c7904353a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:17.769532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-54d10bff-1e26-4ee5-85ed-479c7904353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:17.900295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d10bff-1e26-4ee5-85ed-479c7904353a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:17.904795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68805fce-c4b1-4ea4-a268-25fef4f8a732 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6103e9a612c84a9ea7d21cc5128fa40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:17.905422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68805fce-c4b1-4ea4-a268-25fef4f8a732 service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:17.905694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68805fce-c4b1-4ea4-a268-25fef4f8a732 service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:17.905969 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68805fce-c4b1-4ea4-a268-25fef4f8a732 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:17.917302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68805fce-c4b1-4ea4-a268-25fef4f8a732 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:17.918726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68805fce-c4b1-4ea4-a268-25fef4f8a732 service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:17.931030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1152/2280] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:12:18.017933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4488cfc-d549-4173-baee-60832031e36c None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:18.018210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4488cfc-d549-4173-baee-60832031e36c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:18.018486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4488cfc-d549-4173-baee-60832031e36c None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:18.019968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1129/2281] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 17:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:12:18.026735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8f463-1f37-4835-9b07-22f11607c8f9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:18.028563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8f463-1f37-4835-9b07-22f11607c8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:18.030758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8f463-1f37-4835-9b07-22f11607c8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:18.063901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-64c8f463-1f37-4835-9b07-22f11607c8f9 None None] MFA Rules not processed for user `8c2ed1eb664348e384824f39ca90c950`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:18.065237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-64c8f463-1f37-4835-9b07-22f11607c8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:18.099158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1153/2282] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 17:12:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:12:18.100607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4488cfc-d549-4173-baee-60832031e36c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:18.126082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4488cfc-d549-4173-baee-60832031e36c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:18.130287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62187229-8e84-4f18-915e-ccc8bcb7a664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:18.131755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187229-8e84-4f18-915e-ccc8bcb7a664 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:18.132142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187229-8e84-4f18-915e-ccc8bcb7a664 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:18.132494 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187229-8e84-4f18-915e-ccc8bcb7a664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:18.141478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62187229-8e84-4f18-915e-ccc8bcb7a664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:18.143165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62187229-8e84-4f18-915e-ccc8bcb7a664 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:18.151863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1130/2283] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:12:18.237797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d137470-bfed-4540-828f-f29f24fac596 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:18.238262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d137470-bfed-4540-828f-f29f24fac596 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:18.238692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d137470-bfed-4540-828f-f29f24fac596 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:18.240083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1154/2284] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 17:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:12:18.246178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0118b-522b-4db5-9266-84bc6128561e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:18.246458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0118b-522b-4db5-9266-84bc6128561e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:18.246727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0118b-522b-4db5-9266-84bc6128561e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:18.268374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-88a0118b-522b-4db5-9266-84bc6128561e None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:18.269554 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-88a0118b-522b-4db5-9266-84bc6128561e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:18.279754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1131/2285] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 17:12:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:12:18.291450 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d137470-bfed-4540-828f-f29f24fac596 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:18.305282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d137470-bfed-4540-828f-f29f24fac596 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:18.309308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2f30b-4bb3-4a0a-9b83-7c33bc01373e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:18.309910 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2f30b-4bb3-4a0a-9b83-7c33bc01373e service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:18.310262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2f30b-4bb3-4a0a-9b83-7c33bc01373e service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:18.314652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2f30b-4bb3-4a0a-9b83-7c33bc01373e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:18.354242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2f30b-4bb3-4a0a-9b83-7c33bc01373e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:18.356162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2f30b-4bb3-4a0a-9b83-7c33bc01373e service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:18.369131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1155/2286] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:12:22.495115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76335d3b-b2fa-4713-a9fe-ea29c2e37580 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:22.495639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76335d3b-b2fa-4713-a9fe-ea29c2e37580 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:22.495736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76335d3b-b2fa-4713-a9fe-ea29c2e37580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:22.517895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-76335d3b-b2fa-4713-a9fe-ea29c2e37580 None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:22.519139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-76335d3b-b2fa-4713-a9fe-ea29c2e37580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:22.527171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1132/2287] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:12:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:12:22.539633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2f30b-4bb3-4a0a-9b83-7c33bc01373e service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:22.551093 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2f30b-4bb3-4a0a-9b83-7c33bc01373e service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:22.557030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd85a151-2234-456f-b329-302c5b1f15b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:22.558138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85a151-2234-456f-b329-302c5b1f15b2 service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:22.558555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85a151-2234-456f-b329-302c5b1f15b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:22.558839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85a151-2234-456f-b329-302c5b1f15b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:22.568996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd85a151-2234-456f-b329-302c5b1f15b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:22.570649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd85a151-2234-456f-b329-302c5b1f15b2 service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:22.579418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1156/2288] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:12:24.437757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76335d3b-b2fa-4713-a9fe-ea29c2e37580 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:24.439369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-76335d3b-b2fa-4713-a9fe-ea29c2e37580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:24.541319 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76335d3b-b2fa-4713-a9fe-ea29c2e37580 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:24.551406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad67ec7-2779-46cf-ba56-70db20806b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:24.552071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad67ec7-2779-46cf-ba56-70db20806b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:24.552351 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad67ec7-2779-46cf-ba56-70db20806b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:24.552665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad67ec7-2779-46cf-ba56-70db20806b62 admin admin] PATH_INFO: `/v3/users/4386cce77ec047f480e9999aeb103da5` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:24.554408 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad67ec7-2779-46cf-ba56-70db20806b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4386cce77ec047f480e9999aeb103da5)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:24.555913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad67ec7-2779-46cf-ba56-70db20806b62 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:24.606315 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ad67ec7-2779-46cf-ba56-70db20806b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4386cce77ec047f480e9999aeb103da5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:12:24.620904 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ad67ec7-2779-46cf-ba56-70db20806b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4386cce77ec047f480e9999aeb103da5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:12:24.630522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ad67ec7-2779-46cf-ba56-70db20806b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4386cce77ec047f480e9999aeb103da5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:12:24.631455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ad67ec7-2779-46cf-ba56-70db20806b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4386cce77ec047f480e9999aeb103da5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:12:24.645279 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ad67ec7-2779-46cf-ba56-70db20806b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4386cce77ec047f480e9999aeb103da5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:12:24.652509 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1133/2289] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:12:24 2022] DELETE /identity/v3/users/4386cce77ec047f480e9999aeb103da5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:12:24.839948 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd85a151-2234-456f-b329-302c5b1f15b2 service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:24.854083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd85a151-2234-456f-b329-302c5b1f15b2 service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:24.957409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:24.957982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:24.958268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:24.958535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] PATH_INFO: `/v3/projects/78cce6768e1a473f9056d179b605df6d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:24.959875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78cce6768e1a473f9056d179b605df6d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:24.961809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:25.023858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78cce6768e1a473f9056d179b605df6d', 'request_id': 'req-74440c21-7f9e-4b84-897d-2a203b34edc3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:12:25.026282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:12:25.031163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78cce6768e1a473f9056d179b605df6d', 'request_id': 'req-74440c21-7f9e-4b84-897d-2a203b34edc3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:12:25.038293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78cce6768e1a473f9056d179b605df6d', 'request_id': 'req-74440c21-7f9e-4b84-897d-2a203b34edc3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:12:25.050860 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] The token cache is being invalidate because project 78cce6768e1a473f9056d179b605df6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:12:25.051210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:12:25.051814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-74440c21-7f9e-4b84-897d-2a203b34edc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:12:25.054215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1157/2290] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:12:24 2022] DELETE /identity/v3/projects/78cce6768e1a473f9056d179b605df6d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:12:25.058820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc9f16-98f2-488c-be8f-693942cacb8a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:25.059141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc9f16-98f2-488c-be8f-693942cacb8a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:25.059438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc9f16-98f2-488c-be8f-693942cacb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:25.093814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-94fc9f16-98f2-488c-be8f-693942cacb8a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:25.095371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-94fc9f16-98f2-488c-be8f-693942cacb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:25.144203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:12:25.164868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1134/2291] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:12:25.172543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaafb5d-be4f-4696-aef7-982fc3b3d8d9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:25.172836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaafb5d-be4f-4696-aef7-982fc3b3d8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:25.173119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaafb5d-be4f-4696-aef7-982fc3b3d8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:25.196021 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5aaafb5d-be4f-4696-aef7-982fc3b3d8d9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:25.197272 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5aaafb5d-be4f-4696-aef7-982fc3b3d8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:25.206570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1158/2292] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:12:25.210219 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc9f16-98f2-488c-be8f-693942cacb8a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:25.220408 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc9f16-98f2-488c-be8f-693942cacb8a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:25.225912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5e3c8-b773-485c-9d34-f4cb107c710f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:25.225912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5e3c8-b773-485c-9d34-f4cb107c710f admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:25.225912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5e3c8-b773-485c-9d34-f4cb107c710f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:25.226241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5e3c8-b773-485c-9d34-f4cb107c710f admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:25.226811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5e3c8-b773-485c-9d34-f4cb107c710f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:12:25.227097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5e3c8-b773-485c-9d34-f4cb107c710f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:25.228709 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5e3c8-b773-485c-9d34-f4cb107c710f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:25.241859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1135/2293] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:12:25.247306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaafb5d-be4f-4696-aef7-982fc3b3d8d9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:25.260947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaafb5d-be4f-4696-aef7-982fc3b3d8d9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:25.267308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93eb20-01a3-45f6-aae4-c72fee3faded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:25.268026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93eb20-01a3-45f6-aae4-c72fee3faded admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:25.268309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93eb20-01a3-45f6-aae4-c72fee3faded admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:25.268653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93eb20-01a3-45f6-aae4-c72fee3faded admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:25.269316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93eb20-01a3-45f6-aae4-c72fee3faded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:25.271288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93eb20-01a3-45f6-aae4-c72fee3faded admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:25.301488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1159/2294] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:12:25 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:12:25.309232 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5e3c8-b773-485c-9d34-f4cb107c710f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:25.325170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5e3c8-b773-485c-9d34-f4cb107c710f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:25.397956 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3bd8b-c916-4ab9-a40c-b52e0366c6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:25.398708 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3bd8b-c916-4ab9-a40c-b52e0366c6ec admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:25.399041 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3bd8b-c916-4ab9-a40c-b52e0366c6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:25.399356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3bd8b-c916-4ab9-a40c-b52e0366c6ec admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:25.400004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3bd8b-c916-4ab9-a40c-b52e0366c6ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:25.401956 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3bd8b-c916-4ab9-a40c-b52e0366c6ec admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:25.429208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1136/2295] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:12:25 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:12:25.430967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93eb20-01a3-45f6-aae4-c72fee3faded admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:25.452318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93eb20-01a3-45f6-aae4-c72fee3faded admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:25.462654 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40513a-c270-487d-9bfe-36919d43d063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:25.463687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40513a-c270-487d-9bfe-36919d43d063 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:25.464119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40513a-c270-487d-9bfe-36919d43d063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:25.465255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40513a-c270-487d-9bfe-36919d43d063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:25.466071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40513a-c270-487d-9bfe-36919d43d063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:25.469182 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40513a-c270-487d-9bfe-36919d43d063 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:25.487588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1160/2296] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:12:25.491522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3bd8b-c916-4ab9-a40c-b52e0366c6ec admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:25.510324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3bd8b-c916-4ab9-a40c-b52e0366c6ec admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:25.515237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f9d00-58a4-45a0-84fc-c7d38e5929c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:25.515836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f9d00-58a4-45a0-84fc-c7d38e5929c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:25.516112 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f9d00-58a4-45a0-84fc-c7d38e5929c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:25.516381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f9d00-58a4-45a0-84fc-c7d38e5929c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:25.517014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f9d00-58a4-45a0-84fc-c7d38e5929c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:25.518599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f9d00-58a4-45a0-84fc-c7d38e5929c3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:25.530508 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1137/2297] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:12:25.534813 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40513a-c270-487d-9bfe-36919d43d063 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:25.545587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40513a-c270-487d-9bfe-36919d43d063 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:25.550542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215ba0f-002a-451c-bfb3-2aa1173406bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:25.551290 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215ba0f-002a-451c-bfb3-2aa1173406bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:25.551568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215ba0f-002a-451c-bfb3-2aa1173406bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:25.551838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215ba0f-002a-451c-bfb3-2aa1173406bd admin admin] PATH_INFO: `/v3/projects/0749b2616b854791aec389d669ef94c8/users/cf878d3df653463caa7767de835dce83/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:25.568050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215ba0f-002a-451c-bfb3-2aa1173406bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0749b2616b854791aec389d669ef94c8, user_id=cf878d3df653463caa7767de835dce83, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:25.571310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215ba0f-002a-451c-bfb3-2aa1173406bd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:25.585901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1161/2298] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:12:25 2022] PUT /identity/v3/projects/0749b2616b854791aec389d669ef94c8/users/cf878d3df653463caa7767de835dce83/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:12:25.591280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c4ff8-6517-48a0-9e80-9213e7f9d803 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:25.591618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c4ff8-6517-48a0-9e80-9213e7f9d803 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:25.591894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c4ff8-6517-48a0-9e80-9213e7f9d803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:25.614233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a93c4ff8-6517-48a0-9e80-9213e7f9d803 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:25.615660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a93c4ff8-6517-48a0-9e80-9213e7f9d803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:25.689532 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1138/2299] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:12:25.701110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215ba0f-002a-451c-bfb3-2aa1173406bd admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:25.703385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1215ba0f-002a-451c-bfb3-2aa1173406bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:25.794058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215ba0f-002a-451c-bfb3-2aa1173406bd admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:25.797894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f22c1-d927-4ba1-b90f-0cd97e328a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:25.798510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f22c1-d927-4ba1-b90f-0cd97e328a0e service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:25.798788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f22c1-d927-4ba1-b90f-0cd97e328a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:25.799360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f22c1-d927-4ba1-b90f-0cd97e328a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:25.809162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f22c1-d927-4ba1-b90f-0cd97e328a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:25.810760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f22c1-d927-4ba1-b90f-0cd97e328a0e service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:25.822092 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1162/2300] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:12:29.955950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c4ff8-6517-48a0-9e80-9213e7f9d803 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:29.960661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a93c4ff8-6517-48a0-9e80-9213e7f9d803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:30.096376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:12:30.167298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c4ff8-6517-48a0-9e80-9213e7f9d803 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:30.191871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f136c2-1fc4-4f55-8255-83362e297e7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:30.192457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f136c2-1fc4-4f55-8255-83362e297e7c service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:30.192738 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f136c2-1fc4-4f55-8255-83362e297e7c service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:30.193062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f136c2-1fc4-4f55-8255-83362e297e7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:30.201033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-91f136c2-1fc4-4f55-8255-83362e297e7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:30.372322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f136c2-1fc4-4f55-8255-83362e297e7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:30.373857 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f136c2-1fc4-4f55-8255-83362e297e7c service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:30.395870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1139/2301] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:12:31.716824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97f6d3-be1f-4e7b-a1b5-a82905602af7 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:31.717482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97f6d3-be1f-4e7b-a1b5-a82905602af7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:31.717482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97f6d3-be1f-4e7b-a1b5-a82905602af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:31.748134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-db97f6d3-be1f-4e7b-a1b5-a82905602af7 None None] MFA Rules not processed for user `cf878d3df653463caa7767de835dce83`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:31.749643 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-db97f6d3-be1f-4e7b-a1b5-a82905602af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:31.873779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1163/2302] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:12:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:12:31.880582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f136c2-1fc4-4f55-8255-83362e297e7c service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:31.912422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f136c2-1fc4-4f55-8255-83362e297e7c service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:31.922803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27faaee-15fc-4aa9-a660-ae43dec0dbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:31.923547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27faaee-15fc-4aa9-a660-ae43dec0dbbf admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:31.924142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27faaee-15fc-4aa9-a660-ae43dec0dbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:31.924478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27faaee-15fc-4aa9-a660-ae43dec0dbbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:31.925201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27faaee-15fc-4aa9-a660-ae43dec0dbbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:31.928775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27faaee-15fc-4aa9-a660-ae43dec0dbbf admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:31.983116 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1140/2303] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:12:31 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:12:31.983807 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97f6d3-be1f-4e7b-a1b5-a82905602af7 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:32.023027 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97f6d3-be1f-4e7b-a1b5-a82905602af7 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:32.046563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:12:32.173785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec62179-7116-4526-8fed-0907bebf065e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:32.175074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec62179-7116-4526-8fed-0907bebf065e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:32.175594 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec62179-7116-4526-8fed-0907bebf065e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:32.176057 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec62179-7116-4526-8fed-0907bebf065e admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:32.176904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec62179-7116-4526-8fed-0907bebf065e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:32.179592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec62179-7116-4526-8fed-0907bebf065e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:32.211162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1164/2304] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:12:31 2022] POST /identity/v3/users => generated 299 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:12:32.215680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27faaee-15fc-4aa9-a660-ae43dec0dbbf admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:32.235203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27faaee-15fc-4aa9-a660-ae43dec0dbbf admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:32.246619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca2f1c-c0b9-4f44-87d7-207299788a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:32.247589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca2f1c-c0b9-4f44-87d7-207299788a85 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:32.247948 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca2f1c-c0b9-4f44-87d7-207299788a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:32.248304 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca2f1c-c0b9-4f44-87d7-207299788a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:32.249284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca2f1c-c0b9-4f44-87d7-207299788a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:32.251246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca2f1c-c0b9-4f44-87d7-207299788a85 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:32.300936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1141/2305] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:12:32.302314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec62179-7116-4526-8fed-0907bebf065e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:32.319998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec62179-7116-4526-8fed-0907bebf065e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:32.346148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58dbae-1662-46cf-b3f6-5972e3d4aa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:32.346858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58dbae-1662-46cf-b3f6-5972e3d4aa97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:32.356955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58dbae-1662-46cf-b3f6-5972e3d4aa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:32.357370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58dbae-1662-46cf-b3f6-5972e3d4aa97 admin admin] PATH_INFO: `/v3/projects/717ff3c230e549f684162b5fed430228/users/b34f97f989d44b879db0854801e272c9/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:32.385651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58dbae-1662-46cf-b3f6-5972e3d4aa97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=717ff3c230e549f684162b5fed430228, user_id=b34f97f989d44b879db0854801e272c9, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:32.388408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58dbae-1662-46cf-b3f6-5972e3d4aa97 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:32.416770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca2f1c-c0b9-4f44-87d7-207299788a85 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:32.417849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1165/2306] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:12:32 2022] PUT /identity/v3/projects/717ff3c230e549f684162b5fed430228/users/b34f97f989d44b879db0854801e272c9/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:12:32.429727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca2f1c-c0b9-4f44-87d7-207299788a85 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:32.535130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6197b9-f2a5-403d-b943-3a95138f774c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:32.535643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6197b9-f2a5-403d-b943-3a95138f774c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:32.535891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6197b9-f2a5-403d-b943-3a95138f774c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:32.536165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6197b9-f2a5-403d-b943-3a95138f774c admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:32.540128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6197b9-f2a5-403d-b943-3a95138f774c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:32.541439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6197b9-f2a5-403d-b943-3a95138f774c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:32.579907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1142/2307] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:12:32.585192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58dbae-1662-46cf-b3f6-5972e3d4aa97 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:32.607827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58dbae-1662-46cf-b3f6-5972e3d4aa97 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:32.622430 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8612dee-cca4-4ab5-ab9b-72f0c6eee33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:32.623201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8612dee-cca4-4ab5-ab9b-72f0c6eee33b admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:32.623487 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8612dee-cca4-4ab5-ab9b-72f0c6eee33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:32.623774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8612dee-cca4-4ab5-ab9b-72f0c6eee33b admin admin] PATH_INFO: `/v3/projects/717ff3c230e549f684162b5fed430228/users/b34f97f989d44b879db0854801e272c9/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:32.629750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8612dee-cca4-4ab5-ab9b-72f0c6eee33b admin admin] RBAC: Authorizing `identity:create_grant(project_id=717ff3c230e549f684162b5fed430228, user_id=b34f97f989d44b879db0854801e272c9, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:32.633069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8612dee-cca4-4ab5-ab9b-72f0c6eee33b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:32.661504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1166/2308] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:12:32 2022] PUT /identity/v3/projects/717ff3c230e549f684162b5fed430228/users/b34f97f989d44b879db0854801e272c9/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:12:38.619063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7c67d-8b1e-47fc-a431-0ba13b95cd52 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:38.619713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7c67d-8b1e-47fc-a431-0ba13b95cd52 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:38.619713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7c67d-8b1e-47fc-a431-0ba13b95cd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:38.649639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f0f7c67d-8b1e-47fc-a431-0ba13b95cd52 None None] MFA Rules not processed for user `b34f97f989d44b879db0854801e272c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:38.651008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f0f7c67d-8b1e-47fc-a431-0ba13b95cd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:38.697198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:12:38.727863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1143/2309] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:12:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:12:38.740865 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8612dee-cca4-4ab5-ab9b-72f0c6eee33b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:38.743457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c8612dee-cca4-4ab5-ab9b-72f0c6eee33b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:38.825155 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8612dee-cca4-4ab5-ab9b-72f0c6eee33b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:38.829712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fd4f4-c449-4486-87c7-527d31c624f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6103e9a612c84a9ea7d21cc5128fa40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:38.830331 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fd4f4-c449-4486-87c7-527d31c624f8 service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:38.830610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fd4f4-c449-4486-87c7-527d31c624f8 service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:38.830915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fd4f4-c449-4486-87c7-527d31c624f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:38.919107 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fd4f4-c449-4486-87c7-527d31c624f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:38.921418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fd4f4-c449-4486-87c7-527d31c624f8 service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:38.954327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1167/2310] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:12:39.144844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b68181-9f87-4eb3-99a7-27c37c81ce90 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:39.145129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b68181-9f87-4eb3-99a7-27c37c81ce90 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:39.145404 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b68181-9f87-4eb3-99a7-27c37c81ce90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:39.169977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b2b68181-9f87-4eb3-99a7-27c37c81ce90 None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:39.171214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b2b68181-9f87-4eb3-99a7-27c37c81ce90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:39.253003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1144/2311] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:12:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:12:39.272469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fd4f4-c449-4486-87c7-527d31c624f8 service glance] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:39.274291 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e14fd4f4-c449-4486-87c7-527d31c624f8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:39.297814 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:12:39.362747 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fd4f4-c449-4486-87c7-527d31c624f8 service glance] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:39.369700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfd336-f11d-4db0-a952-470495901d61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:39.371716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfd336-f11d-4db0-a952-470495901d61 service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:39.371716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfd336-f11d-4db0-a952-470495901d61 service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:39.371716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfd336-f11d-4db0-a952-470495901d61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:39.381707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfd336-f11d-4db0-a952-470495901d61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:39.383585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfd336-f11d-4db0-a952-470495901d61 service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:39.396711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1168/2312] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:12:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:12:45.013017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b68181-9f87-4eb3-99a7-27c37c81ce90 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:45.014459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b2b68181-9f87-4eb3-99a7-27c37c81ce90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:45.107125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b68181-9f87-4eb3-99a7-27c37c81ce90 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:45.110857 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfe6a0-eefc-4c07-b2fa-25ac6f2439f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:45.111567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfe6a0-eefc-4c07-b2fa-25ac6f2439f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:45.111843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfe6a0-eefc-4c07-b2fa-25ac6f2439f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:45.112110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfe6a0-eefc-4c07-b2fa-25ac6f2439f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:45.113989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2dcfe6a0-eefc-4c07-b2fa-25ac6f2439f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:45.132074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfe6a0-eefc-4c07-b2fa-25ac6f2439f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:45.133547 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfe6a0-eefc-4c07-b2fa-25ac6f2439f6 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:45.147647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1145/2313] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:12:49.167005 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d302-fdae-47ed-a46f-e68dd3aa8b62 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:49.167972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d302-fdae-47ed-a46f-e68dd3aa8b62 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:49.167972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d302-fdae-47ed-a46f-e68dd3aa8b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:49.188039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d151d302-fdae-47ed-a46f-e68dd3aa8b62 None None] MFA Rules not processed for user `cf878d3df653463caa7767de835dce83`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:49.189277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d151d302-fdae-47ed-a46f-e68dd3aa8b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:49.197199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1169/2314] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:12:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:12:49.507359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfe6a0-eefc-4c07-b2fa-25ac6f2439f6 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:49.516029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfe6a0-eefc-4c07-b2fa-25ac6f2439f6 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:49.519780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8685b89-a061-4ea5-8df6-a2848ed2929a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:49.521067 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8685b89-a061-4ea5-8df6-a2848ed2929a service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:49.521353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8685b89-a061-4ea5-8df6-a2848ed2929a service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:49.521631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8685b89-a061-4ea5-8df6-a2848ed2929a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:49.523409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a8685b89-a061-4ea5-8df6-a2848ed2929a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:49.533497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8685b89-a061-4ea5-8df6-a2848ed2929a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:49.535146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8685b89-a061-4ea5-8df6-a2848ed2929a service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:49.543632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1146/2315] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:12:49.946752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151d302-fdae-47ed-a46f-e68dd3aa8b62 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:49.948279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d151d302-fdae-47ed-a46f-e68dd3aa8b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:50.035911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151d302-fdae-47ed-a46f-e68dd3aa8b62 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:50.041538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4aec6-e97b-4fe4-80f2-a87b4612f419 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:50.042170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4aec6-e97b-4fe4-80f2-a87b4612f419 service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:50.042434 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4aec6-e97b-4fe4-80f2-a87b4612f419 service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:50.042708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4aec6-e97b-4fe4-80f2-a87b4612f419 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:50.044460 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-89e4aec6-e97b-4fe4-80f2-a87b4612f419 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:50.046443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8685b89-a061-4ea5-8df6-a2848ed2929a service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:50.055852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8685b89-a061-4ea5-8df6-a2848ed2929a service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:50.057026 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4aec6-e97b-4fe4-80f2-a87b4612f419 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:50.058607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4aec6-e97b-4fe4-80f2-a87b4612f419 service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:50.059467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800f728-3a0f-472f-825b-933a3a5473b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:50.060054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800f728-3a0f-472f-825b-933a3a5473b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:50.060330 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800f728-3a0f-472f-825b-933a3a5473b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:50.060600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800f728-3a0f-472f-825b-933a3a5473b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:50.062289 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a800f728-3a0f-472f-825b-933a3a5473b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:50.069734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1170/2316] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:12:50.128480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800f728-3a0f-472f-825b-933a3a5473b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:50.130537 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800f728-3a0f-472f-825b-933a3a5473b5 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:50.141005 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1147/2317] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:12:51.949644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d1173-b7ac-489d-9467-9e935bdb3413 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:51.950577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d1173-b7ac-489d-9467-9e935bdb3413 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:51.951300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d1173-b7ac-489d-9467-9e935bdb3413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:51.983306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-525d1173-b7ac-489d-9467-9e935bdb3413 None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:12:51.992615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-525d1173-b7ac-489d-9467-9e935bdb3413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:52.011816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1171/2318] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:12:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:12:52.023361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800f728-3a0f-472f-825b-933a3a5473b5 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:52.038978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800f728-3a0f-472f-825b-933a3a5473b5 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:52.045429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959b0c1-5e65-40ee-8423-59b052e477e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:52.046802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959b0c1-5e65-40ee-8423-59b052e477e0 service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:52.047216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959b0c1-5e65-40ee-8423-59b052e477e0 service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:52.047821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959b0c1-5e65-40ee-8423-59b052e477e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:52.054218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:12:52.058909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959b0c1-5e65-40ee-8423-59b052e477e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:52.061600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959b0c1-5e65-40ee-8423-59b052e477e0 service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:52.071392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1148/2319] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:12:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:12:52.819248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d1173-b7ac-489d-9467-9e935bdb3413 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:12:52.931671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d1173-b7ac-489d-9467-9e935bdb3413 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:12:52.941727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bb212-9298-4cd3-bb6d-b95f774dd0f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:12:52.943823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bb212-9298-4cd3-bb6d-b95f774dd0f8 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:12:52.944122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bb212-9298-4cd3-bb6d-b95f774dd0f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:12:52.944448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bb212-9298-4cd3-bb6d-b95f774dd0f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:12:52.946861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-699bb212-9298-4cd3-bb6d-b95f774dd0f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:12:52.977003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bb212-9298-4cd3-bb6d-b95f774dd0f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:12:52.981406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bb212-9298-4cd3-bb6d-b95f774dd0f8 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:12:53.004154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1172/2320] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:12:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:13:02.067585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:13:02.997274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:13:12.954291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959b0c1-5e65-40ee-8423-59b052e477e0 service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:13:12.955921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d959b0c1-5e65-40ee-8423-59b052e477e0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:13:12.971314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959b0c1-5e65-40ee-8423-59b052e477e0 service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:13:12.978070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46335f-5ad7-4a43-96f8-85a78a554e6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:13:12.978686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46335f-5ad7-4a43-96f8-85a78a554e6e service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:13:12.978981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46335f-5ad7-4a43-96f8-85a78a554e6e service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:13:12.979259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46335f-5ad7-4a43-96f8-85a78a554e6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:13:12.980983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fe46335f-5ad7-4a43-96f8-85a78a554e6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:13:13.043011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46335f-5ad7-4a43-96f8-85a78a554e6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:13:13.044627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46335f-5ad7-4a43-96f8-85a78a554e6e service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:13:13.055666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1149/2321] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:13:12 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:13:13.106974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bede6d-ad0c-4b35-acd4-a4bfc31776f1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:13:13.107393 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bede6d-ad0c-4b35-acd4-a4bfc31776f1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:13:13.107531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bede6d-ad0c-4b35-acd4-a4bfc31776f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:13:13.129654 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-46bede6d-ad0c-4b35-acd4-a4bfc31776f1 None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:13:13.130923 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-46bede6d-ad0c-4b35-acd4-a4bfc31776f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:13:13.139377 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1173/2322] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 22 17:13:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:13:13.165478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46335f-5ad7-4a43-96f8-85a78a554e6e service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:13:13.179490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46335f-5ad7-4a43-96f8-85a78a554e6e service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:13:13.184424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec412fbe-a1f6-438b-b31e-52b944c231ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:13:13.185028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec412fbe-a1f6-438b-b31e-52b944c231ef service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:13:13.185309 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec412fbe-a1f6-438b-b31e-52b944c231ef service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:13:13.185843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec412fbe-a1f6-438b-b31e-52b944c231ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:13:13.194840 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec412fbe-a1f6-438b-b31e-52b944c231ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:13:13.196477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec412fbe-a1f6-438b-b31e-52b944c231ef service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:13:13.205215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1150/2323] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:13:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:13:16.919020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bede6d-ad0c-4b35-acd4-a4bfc31776f1 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:13:16.936443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bede6d-ad0c-4b35-acd4-a4bfc31776f1 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:13:16.945010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f050bf-7071-4412-a46c-8452526a7a62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:13:16.946024 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f050bf-7071-4412-a46c-8452526a7a62 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:13:16.946300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f050bf-7071-4412-a46c-8452526a7a62 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:13:16.946568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f050bf-7071-4412-a46c-8452526a7a62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:13:16.948757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c1f050bf-7071-4412-a46c-8452526a7a62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:13:16.963614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f050bf-7071-4412-a46c-8452526a7a62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:13:16.965611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f050bf-7071-4412-a46c-8452526a7a62 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:13:16.978477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1174/2324] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:13:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:13:20.241215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec412fbe-a1f6-438b-b31e-52b944c231ef service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:13:20.254277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec412fbe-a1f6-438b-b31e-52b944c231ef service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:13:20.260672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6d954-ff73-4976-b337-0e2bd25999f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:13:20.261279 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6d954-ff73-4976-b337-0e2bd25999f9 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:13:20.261563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6d954-ff73-4976-b337-0e2bd25999f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:13:20.261839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6d954-ff73-4976-b337-0e2bd25999f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:13:20.263630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ecd6d954-ff73-4976-b337-0e2bd25999f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:13:20.324122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6d954-ff73-4976-b337-0e2bd25999f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:13:20.325766 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6d954-ff73-4976-b337-0e2bd25999f9 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:13:20.336282 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1151/2325] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:13:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:13:26.972810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=9 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:13:30.330821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=62 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:13:32.296681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f050bf-7071-4412-a46c-8452526a7a62 service nova] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:13:32.306906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f050bf-7071-4412-a46c-8452526a7a62 service nova] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:13:32.310482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c3516-0da6-4452-835d-21dc41d3e640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:13:32.311116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c3516-0da6-4452-835d-21dc41d3e640 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:13:32.311434 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c3516-0da6-4452-835d-21dc41d3e640 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:13:32.311704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c3516-0da6-4452-835d-21dc41d3e640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:13:32.321082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c3516-0da6-4452-835d-21dc41d3e640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:13:32.322483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c3516-0da6-4452-835d-21dc41d3e640 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:13:32.331932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1175/2326] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:13:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:13:36.315426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6d954-ff73-4976-b337-0e2bd25999f9 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:13:36.375013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6d954-ff73-4976-b337-0e2bd25999f9 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:13:36.383674 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaca44a-e144-4b5a-bbd6-0c63f90793a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:13:36.384304 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaca44a-e144-4b5a-bbd6-0c63f90793a5 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:13:36.384579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaca44a-e144-4b5a-bbd6-0c63f90793a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:13:36.384849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaca44a-e144-4b5a-bbd6-0c63f90793a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:13:36.386892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-beaca44a-e144-4b5a-bbd6-0c63f90793a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:13:36.430363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaca44a-e144-4b5a-bbd6-0c63f90793a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:13:36.431890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaca44a-e144-4b5a-bbd6-0c63f90793a5 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:13:36.447153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1152/2327] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:13:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:13:42.327865 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=6 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:13:44.599918 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a60813-e11a-491b-9fe5-4b224e03e92a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:13:44.600528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a60813-e11a-491b-9fe5-4b224e03e92a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:13:44.600528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a60813-e11a-491b-9fe5-4b224e03e92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:13:44.620836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-73a60813-e11a-491b-9fe5-4b224e03e92a None None] MFA Rules not processed for user `bc8c2a5413c943ed94d76eceb2720f17`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:13:44.622041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-73a60813-e11a-491b-9fe5-4b224e03e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:13:44.629569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1176/2328] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:13:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:13:46.438654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=28 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:13:52.761520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaca44a-e144-4b5a-bbd6-0c63f90793a5 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:13:52.772629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaca44a-e144-4b5a-bbd6-0c63f90793a5 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:13:52.778058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0993489-12ce-4cf8-839c-113288c57a9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:13:52.778648 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0993489-12ce-4cf8-839c-113288c57a9c service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:13:52.778976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0993489-12ce-4cf8-839c-113288c57a9c service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:13:52.779252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0993489-12ce-4cf8-839c-113288c57a9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:13:52.780983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f0993489-12ce-4cf8-839c-113288c57a9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:13:52.840900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0993489-12ce-4cf8-839c-113288c57a9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:13:52.842484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0993489-12ce-4cf8-839c-113288c57a9c service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:13:52.852159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1153/2329] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:13:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:13:54.616015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:14:02.847416 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=28 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:14:06.958709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c4cc8-bcbf-4bc6-9cb2-230aeff9faf1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:14:06.959260 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c4cc8-bcbf-4bc6-9cb2-230aeff9faf1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:14:06.959367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c4cc8-bcbf-4bc6-9cb2-230aeff9faf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:14:06.978772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ea7c4cc8-bcbf-4bc6-9cb2-230aeff9faf1 None None] MFA Rules not processed for user `5feea3b3b7e4401ea7db242cb60fdfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:14:06.980099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ea7c4cc8-bcbf-4bc6-9cb2-230aeff9faf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:14:06.987857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1177/2330] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:14:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:14:07.301907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0993489-12ce-4cf8-839c-113288c57a9c service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:14:07.320673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0993489-12ce-4cf8-839c-113288c57a9c service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:14:07.324324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a2cb3-3169-4f8d-8490-8833b4c7f6c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:14:07.324924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a2cb3-3169-4f8d-8490-8833b4c7f6c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:14:07.325207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a2cb3-3169-4f8d-8490-8833b4c7f6c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:14:07.325483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a2cb3-3169-4f8d-8490-8833b4c7f6c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:14:07.327267 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1b2a2cb3-3169-4f8d-8490-8833b4c7f6c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:14:07.353183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a2cb3-3169-4f8d-8490-8833b4c7f6c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:14:07.354629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a2cb3-3169-4f8d-8490-8833b4c7f6c1 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:14:07.363551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1154/2331] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:14:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:14:16.973339 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:14:17.359236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=15 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:14:20.648705 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c4cc8-bcbf-4bc6-9cb2-230aeff9faf1 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:14:20.662349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c4cc8-bcbf-4bc6-9cb2-230aeff9faf1 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:14:20.667851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec850e6b-4c17-49d9-96f6-498712d4ee14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:14:20.668461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec850e6b-4c17-49d9-96f6-498712d4ee14 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:14:20.668746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec850e6b-4c17-49d9-96f6-498712d4ee14 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:14:20.669019 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec850e6b-4c17-49d9-96f6-498712d4ee14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:14:20.670775 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ec850e6b-4c17-49d9-96f6-498712d4ee14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:14:20.730938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec850e6b-4c17-49d9-96f6-498712d4ee14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:14:20.732553 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec850e6b-4c17-49d9-96f6-498712d4ee14 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:14:20.742188 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1178/2332] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:14:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:14:30.737579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=28 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:14:30.906495 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4e887-d1da-4570-a14f-9095efe3698c None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:14:30.907466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4e887-d1da-4570-a14f-9095efe3698c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:14:30.907750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4e887-d1da-4570-a14f-9095efe3698c None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:14:30.909205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1155/2333] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 17:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:14:30.915724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786b890-81ff-485d-acc9-8f9765bacdd6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:14:30.915992 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786b890-81ff-485d-acc9-8f9765bacdd6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:14:30.916266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786b890-81ff-485d-acc9-8f9765bacdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:14:30.961905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b786b890-81ff-485d-acc9-8f9765bacdd6 None None] MFA Rules not processed for user `8c2ed1eb664348e384824f39ca90c950`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:14:30.963443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b786b890-81ff-485d-acc9-8f9765bacdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:14:30.970578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1179/2334] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 17:14:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:14:30.985317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4e887-d1da-4570-a14f-9095efe3698c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:14:30.997454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4e887-d1da-4570-a14f-9095efe3698c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:14:31.001352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2596bc5-5616-4ca9-95a4-373c7dcda8f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:14:31.001988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2596bc5-5616-4ca9-95a4-373c7dcda8f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:14:31.002270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2596bc5-5616-4ca9-95a4-373c7dcda8f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:14:31.002577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2596bc5-5616-4ca9-95a4-373c7dcda8f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:14:31.016405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2596bc5-5616-4ca9-95a4-373c7dcda8f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:14:31.019497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2596bc5-5616-4ca9-95a4-373c7dcda8f5 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:14:31.035435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1156/2335] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:14:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:14:31.134595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c21e47-4d8c-4e03-9208-d2ded1c163e4 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:14:31.134884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c21e47-4d8c-4e03-9208-d2ded1c163e4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:14:31.135190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c21e47-4d8c-4e03-9208-d2ded1c163e4 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:14:31.136568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1180/2336] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 17:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:14:31.145071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3bddb-1f15-4eca-98bb-5ca98d97cdbe None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:14:31.145360 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3bddb-1f15-4eca-98bb-5ca98d97cdbe None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:14:31.145676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3bddb-1f15-4eca-98bb-5ca98d97cdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:14:31.166730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-fea3bddb-1f15-4eca-98bb-5ca98d97cdbe None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:14:31.168019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fea3bddb-1f15-4eca-98bb-5ca98d97cdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:14:31.178719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1157/2337] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 17:14:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:14:31.193700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c21e47-4d8c-4e03-9208-d2ded1c163e4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:14:31.210120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c21e47-4d8c-4e03-9208-d2ded1c163e4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:14:31.214500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039e284-f54d-4d88-a027-09b4c2353779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:14:31.215299 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039e284-f54d-4d88-a027-09b4c2353779 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:14:31.215704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039e284-f54d-4d88-a027-09b4c2353779 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:14:31.216181 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039e284-f54d-4d88-a027-09b4c2353779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:14:31.231805 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039e284-f54d-4d88-a027-09b4c2353779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:14:31.234338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039e284-f54d-4d88-a027-09b4c2353779 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:14:31.254371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1181/2338] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:14:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:14:41.161441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=9 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:14:41.245566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=17 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:14:42.383285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe29c15-4430-40b2-8db4-8f2855cbe5d8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:14:42.383840 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe29c15-4430-40b2-8db4-8f2855cbe5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:14:42.384202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe29c15-4430-40b2-8db4-8f2855cbe5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:14:42.414772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bfe29c15-4430-40b2-8db4-8f2855cbe5d8 None None] MFA Rules not processed for user `b34e9318826c49ed891dde1f7782898b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:14:42.416915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bfe29c15-4430-40b2-8db4-8f2855cbe5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:14:42.475755 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1158/2339] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:14:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:14:52.468669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=25 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:10.089129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44f9a-937b-4f59-aeee-f20f8e35e84e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:10.090256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44f9a-937b-4f59-aeee-f20f8e35e84e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:10.090538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44f9a-937b-4f59-aeee-f20f8e35e84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:10.111707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-dfb44f9a-937b-4f59-aeee-f20f8e35e84e None None] MFA Rules not processed for user `624f8fd9a74e48afa2f8b74b090a8dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:10.112899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dfb44f9a-937b-4f59-aeee-f20f8e35e84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:10.120943 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1182/2340] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:19.947562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe29c15-4430-40b2-8db4-8f2855cbe5d8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:19.949013 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bfe29c15-4430-40b2-8db4-8f2855cbe5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:19.991711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe29c15-4430-40b2-8db4-8f2855cbe5d8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:19.998330 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee81e6-1b05-4ce7-8e8e-21d607d64359 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:19.998964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee81e6-1b05-4ce7-8e8e-21d607d64359 service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:19.999257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee81e6-1b05-4ce7-8e8e-21d607d64359 service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:19.999531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee81e6-1b05-4ce7-8e8e-21d607d64359 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:20.001899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-72ee81e6-1b05-4ce7-8e8e-21d607d64359 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:20.015328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee81e6-1b05-4ce7-8e8e-21d607d64359 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:20.016988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee81e6-1b05-4ce7-8e8e-21d607d64359 service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:20.029552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1159/2341] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:20.105663 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:21.281154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb44f9a-937b-4f59-aeee-f20f8e35e84e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:21.291847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb44f9a-937b-4f59-aeee-f20f8e35e84e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:21.295559 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef97b2-5b59-44cc-b306-91cf4d508557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:21.296176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef97b2-5b59-44cc-b306-91cf4d508557 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:21.296441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef97b2-5b59-44cc-b306-91cf4d508557 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:21.296718 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef97b2-5b59-44cc-b306-91cf4d508557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:21.299071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d1ef97b2-5b59-44cc-b306-91cf4d508557 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:21.391824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef97b2-5b59-44cc-b306-91cf4d508557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:21.393183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef97b2-5b59-44cc-b306-91cf4d508557 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:21.413951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1183/2342] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:26.554803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee81e6-1b05-4ce7-8e8e-21d607d64359 service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:26.610044 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee81e6-1b05-4ce7-8e8e-21d607d64359 service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:26.628501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5622da7-c717-4951-9ef8-47afe02d4353 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:26.630157 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622da7-c717-4951-9ef8-47afe02d4353 service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:26.631008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622da7-c717-4951-9ef8-47afe02d4353 service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:26.631713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622da7-c717-4951-9ef8-47afe02d4353 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:26.635509 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d5622da7-c717-4951-9ef8-47afe02d4353 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:26.658563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef97b2-5b59-44cc-b306-91cf4d508557 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:26.661752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d1ef97b2-5b59-44cc-b306-91cf4d508557 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:26.679578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef97b2-5b59-44cc-b306-91cf4d508557 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:26.683014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5622da7-c717-4951-9ef8-47afe02d4353 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:26.687752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5622da7-c717-4951-9ef8-47afe02d4353 service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:26.693261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550144a-9948-4910-90b2-0da0aae8b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:26.694298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550144a-9948-4910-90b2-0da0aae8b29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:26.694858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550144a-9948-4910-90b2-0da0aae8b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:26.695389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550144a-9948-4910-90b2-0da0aae8b29f admin admin] PATH_INFO: `/v3/users/bc8c2a5413c943ed94d76eceb2720f17` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:26.700468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550144a-9948-4910-90b2-0da0aae8b29f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8c2a5413c943ed94d76eceb2720f17)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:26.703221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550144a-9948-4910-90b2-0da0aae8b29f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:26.714388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1160/2343] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:15:26.911886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3550144a-9948-4910-90b2-0da0aae8b29f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8c2a5413c943ed94d76eceb2720f17'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:26.936698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3550144a-9948-4910-90b2-0da0aae8b29f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8c2a5413c943ed94d76eceb2720f17'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:26.946863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3550144a-9948-4910-90b2-0da0aae8b29f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8c2a5413c943ed94d76eceb2720f17'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:26.959590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3550144a-9948-4910-90b2-0da0aae8b29f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8c2a5413c943ed94d76eceb2720f17'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:26.960901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3550144a-9948-4910-90b2-0da0aae8b29f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8c2a5413c943ed94d76eceb2720f17'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:26.972927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1184/2344] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:26 2022] DELETE /identity/v3/users/bc8c2a5413c943ed94d76eceb2720f17 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:15:27.223591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5622da7-c717-4951-9ef8-47afe02d4353 service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:27.249525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5622da7-c717-4951-9ef8-47afe02d4353 service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:27.345960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88b73d-f171-4a17-9deb-9f5c9e13e8c8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:27.346263 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88b73d-f171-4a17-9deb-9f5c9e13e8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:27.346538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88b73d-f171-4a17-9deb-9f5c9e13e8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:27.373974 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:27.376710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:27.377502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:27.378206 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] PATH_INFO: `/v3/projects/105c11557b6f45329dc97af01e8ea915` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:27.381340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=105c11557b6f45329dc97af01e8ea915)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:27.385069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8a88b73d-f171-4a17-9deb-9f5c9e13e8c8 None None] MFA Rules not processed for user `c2060b33d0a24e01a96ab3fb02b4b045`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:27.386469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8a88b73d-f171-4a17-9deb-9f5c9e13e8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:27.386738 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:27.431850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:27.471984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '105c11557b6f45329dc97af01e8ea915', 'request_id': 'req-2a63ee3e-2615-44a1-b656-238e16da30b9'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:27.489106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '105c11557b6f45329dc97af01e8ea915', 'request_id': 'req-2a63ee3e-2615-44a1-b656-238e16da30b9'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:27.503101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '105c11557b6f45329dc97af01e8ea915', 'request_id': 'req-2a63ee3e-2615-44a1-b656-238e16da30b9'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:27.510859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1185/2345] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:27.551182 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] The token cache is being invalidate because project 105c11557b6f45329dc97af01e8ea915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:27.551750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:27.552647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2a63ee3e-2615-44a1-b656-238e16da30b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:27.554429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1161/2346] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:27 2022] DELETE /identity/v3/projects/105c11557b6f45329dc97af01e8ea915 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:27.561038 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8eb5b-1574-49e1-bf68-d82a5715f24c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:27.561520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8eb5b-1574-49e1-bf68-d82a5715f24c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:27.562033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8eb5b-1574-49e1-bf68-d82a5715f24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:27.706468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-dee8eb5b-1574-49e1-bf68-d82a5715f24c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:27.711876 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dee8eb5b-1574-49e1-bf68-d82a5715f24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:27.803668 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1186/2347] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:27.812576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455e95d-dad3-4e1d-991e-61a57e691374 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:27.813927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455e95d-dad3-4e1d-991e-61a57e691374 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:27.813927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455e95d-dad3-4e1d-991e-61a57e691374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:27.836886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d455e95d-dad3-4e1d-991e-61a57e691374 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:27.838225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d455e95d-dad3-4e1d-991e-61a57e691374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:27.847892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1162/2348] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:27.852892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8eb5b-1574-49e1-bf68-d82a5715f24c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:27.871687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8eb5b-1574-49e1-bf68-d82a5715f24c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:27.880804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebb1be-c661-4658-b9de-941f148aa8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:27.881439 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebb1be-c661-4658-b9de-941f148aa8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:27.882153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebb1be-c661-4658-b9de-941f148aa8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:27.882153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebb1be-c661-4658-b9de-941f148aa8a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:27.882851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebb1be-c661-4658-b9de-941f148aa8a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:15:27.883186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebb1be-c661-4658-b9de-941f148aa8a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:27.884849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebb1be-c661-4658-b9de-941f148aa8a5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:27.901104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1187/2349] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:15:27.907546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455e95d-dad3-4e1d-991e-61a57e691374 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:27.920725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455e95d-dad3-4e1d-991e-61a57e691374 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:27.926807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d87d0-5b1d-436b-a718-cb01a362f0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:27.928161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d87d0-5b1d-436b-a718-cb01a362f0fe admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:27.928690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d87d0-5b1d-436b-a718-cb01a362f0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:27.929192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d87d0-5b1d-436b-a718-cb01a362f0fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:27.930105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d87d0-5b1d-436b-a718-cb01a362f0fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:27.932743 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d87d0-5b1d-436b-a718-cb01a362f0fe admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:27.990894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1163/2350] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:27 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:27.996864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebb1be-c661-4658-b9de-941f148aa8a5 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:28.017960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebb1be-c661-4658-b9de-941f148aa8a5 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:28.105344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df6751-6003-47e0-bbe8-f656c894ac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:28.106846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df6751-6003-47e0-bbe8-f656c894ac79 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:28.106846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df6751-6003-47e0-bbe8-f656c894ac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:28.106846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df6751-6003-47e0-bbe8-f656c894ac79 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:28.107414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df6751-6003-47e0-bbe8-f656c894ac79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:28.110175 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df6751-6003-47e0-bbe8-f656c894ac79 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:28.152778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1188/2351] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:27 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:28.154110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d87d0-5b1d-436b-a718-cb01a362f0fe admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:28.166824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d87d0-5b1d-436b-a718-cb01a362f0fe admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:28.173589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686cfb9-1191-495e-a56a-fa9857ee7e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:28.174988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686cfb9-1191-495e-a56a-fa9857ee7e7d admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:28.175499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686cfb9-1191-495e-a56a-fa9857ee7e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:28.175954 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686cfb9-1191-495e-a56a-fa9857ee7e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:28.176796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686cfb9-1191-495e-a56a-fa9857ee7e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:28.178839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686cfb9-1191-495e-a56a-fa9857ee7e7d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:28.180984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:28.195823 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1164/2352] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:15:28.200424 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df6751-6003-47e0-bbe8-f656c894ac79 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:28.215615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df6751-6003-47e0-bbe8-f656c894ac79 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:28.221861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c0a41-7ecc-4af3-8a06-f7945e548792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:28.222566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c0a41-7ecc-4af3-8a06-f7945e548792 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:28.222845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c0a41-7ecc-4af3-8a06-f7945e548792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:28.223202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c0a41-7ecc-4af3-8a06-f7945e548792 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:28.223906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c0a41-7ecc-4af3-8a06-f7945e548792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:28.225616 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c0a41-7ecc-4af3-8a06-f7945e548792 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:28.239085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1189/2353] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:15:28.243914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686cfb9-1191-495e-a56a-fa9857ee7e7d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:28.263287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686cfb9-1191-495e-a56a-fa9857ee7e7d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:28.272612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cc15b-05eb-49f2-97e3-a9b0c65a9c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:28.273759 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc15b-05eb-49f2-97e3-a9b0c65a9c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:28.274238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc15b-05eb-49f2-97e3-a9b0c65a9c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:28.274706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc15b-05eb-49f2-97e3-a9b0c65a9c9e admin admin] PATH_INFO: `/v3/projects/d5ffaded03d84792a5743c28143b5e56/users/53008814bbc742a9b0fc69c017b52638/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:28.307969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cc15b-05eb-49f2-97e3-a9b0c65a9c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ffaded03d84792a5743c28143b5e56, user_id=53008814bbc742a9b0fc69c017b52638, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:28.313747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cc15b-05eb-49f2-97e3-a9b0c65a9c9e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:28.362989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1165/2354] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:28 2022] PUT /identity/v3/projects/d5ffaded03d84792a5743c28143b5e56/users/53008814bbc742a9b0fc69c017b52638/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:28.369366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62058f5a-ab99-4795-9b39-814ef76d6d8e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:28.369366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62058f5a-ab99-4795-9b39-814ef76d6d8e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:28.369366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62058f5a-ab99-4795-9b39-814ef76d6d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:28.410890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-62058f5a-ab99-4795-9b39-814ef76d6d8e None None] MFA Rules not processed for user `53008814bbc742a9b0fc69c017b52638`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:28.413149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-62058f5a-ab99-4795-9b39-814ef76d6d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:28.430727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:28.521983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1190/2355] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:15:28.528738 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cc15b-05eb-49f2-97e3-a9b0c65a9c9e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:28.648640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cc15b-05eb-49f2-97e3-a9b0c65a9c9e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:28.653640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c5c35-d333-41a6-a108-8f16b1aac456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:28.654280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c5c35-d333-41a6-a108-8f16b1aac456 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:28.654556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c5c35-d333-41a6-a108-8f16b1aac456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:28.655034 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c5c35-d333-41a6-a108-8f16b1aac456 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:28.655683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c5c35-d333-41a6-a108-8f16b1aac456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:28.658162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c5c35-d333-41a6-a108-8f16b1aac456 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:28.702413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1166/2356] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:28 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:28.704461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62058f5a-ab99-4795-9b39-814ef76d6d8e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:28.721608 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62058f5a-ab99-4795-9b39-814ef76d6d8e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:28.867586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f35fb6-edc2-4fe0-bca4-7be51da4ed83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:28.868684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f35fb6-edc2-4fe0-bca4-7be51da4ed83 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:28.869158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f35fb6-edc2-4fe0-bca4-7be51da4ed83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:28.869631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f35fb6-edc2-4fe0-bca4-7be51da4ed83 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:28.870838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f35fb6-edc2-4fe0-bca4-7be51da4ed83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:28.875384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f35fb6-edc2-4fe0-bca4-7be51da4ed83 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:28.915569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1191/2357] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:28 2022] POST /identity/v3/users => generated 300 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:28.919551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c5c35-d333-41a6-a108-8f16b1aac456 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:28.939382 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c5c35-d333-41a6-a108-8f16b1aac456 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:28.950723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801a8a1-60c8-4f77-9603-b4beb9aef6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:28.951794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801a8a1-60c8-4f77-9603-b4beb9aef6cc admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:28.952498 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801a8a1-60c8-4f77-9603-b4beb9aef6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:28.953043 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801a8a1-60c8-4f77-9603-b4beb9aef6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:28.954559 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801a8a1-60c8-4f77-9603-b4beb9aef6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:28.957332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801a8a1-60c8-4f77-9603-b4beb9aef6cc admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:28.978478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1167/2358] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:28.984147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f35fb6-edc2-4fe0-bca4-7be51da4ed83 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:29.000739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f35fb6-edc2-4fe0-bca4-7be51da4ed83 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:29.007675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346da20c-fe8c-46f4-a682-7f38b1cd3009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:29.008786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346da20c-fe8c-46f4-a682-7f38b1cd3009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:29.010123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346da20c-fe8c-46f4-a682-7f38b1cd3009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:29.010584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346da20c-fe8c-46f4-a682-7f38b1cd3009 admin admin] PATH_INFO: `/v3/projects/a9e44ec35943451da81a947182c7f106/users/06ae5e2b881545e5bb3b90fe68b4163b/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:29.036338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346da20c-fe8c-46f4-a682-7f38b1cd3009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e44ec35943451da81a947182c7f106, user_id=06ae5e2b881545e5bb3b90fe68b4163b, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:29.042557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346da20c-fe8c-46f4-a682-7f38b1cd3009 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:29.069283 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1192/2359] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:28 2022] PUT /identity/v3/projects/a9e44ec35943451da81a947182c7f106/users/06ae5e2b881545e5bb3b90fe68b4163b/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:29.075622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801a8a1-60c8-4f77-9603-b4beb9aef6cc admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:29.098721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801a8a1-60c8-4f77-9603-b4beb9aef6cc admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:29.207477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd85a1-6878-4427-bb63-ffd50d29a605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:29.208100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd85a1-6878-4427-bb63-ffd50d29a605 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:29.208394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd85a1-6878-4427-bb63-ffd50d29a605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:29.208666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd85a1-6878-4427-bb63-ffd50d29a605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:29.209714 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd85a1-6878-4427-bb63-ffd50d29a605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:29.211108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd85a1-6878-4427-bb63-ffd50d29a605 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:29.227750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1168/2360] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:29.229591 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346da20c-fe8c-46f4-a682-7f38b1cd3009 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:29.242741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346da20c-fe8c-46f4-a682-7f38b1cd3009 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:29.249154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06247b-096a-4e02-99af-cd2a4d37a55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:29.250926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06247b-096a-4e02-99af-cd2a4d37a55f admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:29.251449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06247b-096a-4e02-99af-cd2a4d37a55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:29.252015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06247b-096a-4e02-99af-cd2a4d37a55f admin admin] PATH_INFO: `/v3/projects/a9e44ec35943451da81a947182c7f106/users/06ae5e2b881545e5bb3b90fe68b4163b/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:29.259567 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06247b-096a-4e02-99af-cd2a4d37a55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e44ec35943451da81a947182c7f106, user_id=06ae5e2b881545e5bb3b90fe68b4163b, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:29.262531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06247b-096a-4e02-99af-cd2a4d37a55f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:29.274562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1193/2361] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:29 2022] PUT /identity/v3/projects/a9e44ec35943451da81a947182c7f106/users/06ae5e2b881545e5bb3b90fe68b4163b/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:15:29.287483 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419c68e-8482-4316-a0b5-9a873d2b47bb None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:29.287824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419c68e-8482-4316-a0b5-9a873d2b47bb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:29.288006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419c68e-8482-4316-a0b5-9a873d2b47bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:29.324563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8419c68e-8482-4316-a0b5-9a873d2b47bb None None] MFA Rules not processed for user `06ae5e2b881545e5bb3b90fe68b4163b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:29.325884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8419c68e-8482-4316-a0b5-9a873d2b47bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:29.362382 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:29.410879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1169/2362] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:29.427826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06247b-096a-4e02-99af-cd2a4d37a55f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:29.439966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3b06247b-096a-4e02-99af-cd2a4d37a55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:29.529519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:29.532947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06247b-096a-4e02-99af-cd2a4d37a55f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:29.538903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d6f6a-fc56-42ac-9206-cc4858e84073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:29.539907 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d6f6a-fc56-42ac-9206-cc4858e84073 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:29.540192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d6f6a-fc56-42ac-9206-cc4858e84073 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:29.540556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d6f6a-fc56-42ac-9206-cc4858e84073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:29.550576 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d6f6a-fc56-42ac-9206-cc4858e84073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:29.552066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d6f6a-fc56-42ac-9206-cc4858e84073 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:29.563811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1194/2363] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:29.645069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419c68e-8482-4316-a0b5-9a873d2b47bb None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:29.719154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419c68e-8482-4316-a0b5-9a873d2b47bb None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:29.724515 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:29.725169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:29.725444 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:29.725716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] PATH_INFO: `/v3/users/53008814bbc742a9b0fc69c017b52638` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:29.726858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=53008814bbc742a9b0fc69c017b52638)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:29.728866 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:29.796612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53008814bbc742a9b0fc69c017b52638'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:29.809030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53008814bbc742a9b0fc69c017b52638'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:29.816354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53008814bbc742a9b0fc69c017b52638'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:29.817198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53008814bbc742a9b0fc69c017b52638'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:29.833869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53008814bbc742a9b0fc69c017b52638'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:29.840990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1170/2364] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:29 2022] DELETE /identity/v3/users/53008814bbc742a9b0fc69c017b52638 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:29.846371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a85f5-df64-4341-99e7-287580354dca None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:29.846901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a85f5-df64-4341-99e7-287580354dca None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:29.846901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a85f5-df64-4341-99e7-287580354dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:29.867251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-4e0a85f5-df64-4341-99e7-287580354dca None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:29.868851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4e0a85f5-df64-4341-99e7-287580354dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:29.939771 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1195/2365] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:29.950205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:29.951842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:30.009148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:30.017169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc804456-4325-42de-ab9b-2a7656985a5e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:30.021870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495715f3-7be6-4456-a700-0437cc85889f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:30.022482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495715f3-7be6-4456-a700-0437cc85889f service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:30.022749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495715f3-7be6-4456-a700-0437cc85889f service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:30.023051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495715f3-7be6-4456-a700-0437cc85889f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:30.032956 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495715f3-7be6-4456-a700-0437cc85889f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:30.034398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495715f3-7be6-4456-a700-0437cc85889f service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:30.045294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1171/2366] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:30.413315 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a85f5-df64-4341-99e7-287580354dca None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:30.415394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4e0a85f5-df64-4341-99e7-287580354dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:30.431362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a85f5-df64-4341-99e7-287580354dca None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:30.437249 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:30.438513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:30.438794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:30.439134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] PATH_INFO: `/v3/projects/d5ffaded03d84792a5743c28143b5e56` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:30.441103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ffaded03d84792a5743c28143b5e56)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:30.443546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:30.504029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ffaded03d84792a5743c28143b5e56', 'request_id': 'req-078e446b-75a9-4c9f-843e-4e9a41632980'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:30.513983 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ffaded03d84792a5743c28143b5e56', 'request_id': 'req-078e446b-75a9-4c9f-843e-4e9a41632980'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:30.525217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ffaded03d84792a5743c28143b5e56', 'request_id': 'req-078e446b-75a9-4c9f-843e-4e9a41632980'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:30.546498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] The token cache is being invalidate because project d5ffaded03d84792a5743c28143b5e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:30.548139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:30.549452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:30.552300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1196/2367] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:30 2022] DELETE /identity/v3/projects/d5ffaded03d84792a5743c28143b5e56 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:30.558597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495715f3-7be6-4456-a700-0437cc85889f service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:30.586278 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495715f3-7be6-4456-a700-0437cc85889f service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:30.650088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065af82-7ae8-485f-888c-63972cc90a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:30.650574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065af82-7ae8-485f-888c-63972cc90a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:30.650847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065af82-7ae8-485f-888c-63972cc90a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:30.651146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065af82-7ae8-485f-888c-63972cc90a61 admin admin] PATH_INFO: `/v3/users/06ae5e2b881545e5bb3b90fe68b4163b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:30.652277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065af82-7ae8-485f-888c-63972cc90a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ae5e2b881545e5bb3b90fe68b4163b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:30.654204 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065af82-7ae8-485f-888c-63972cc90a61 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:30.726832 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3065af82-7ae8-485f-888c-63972cc90a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ae5e2b881545e5bb3b90fe68b4163b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:30.742178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3065af82-7ae8-485f-888c-63972cc90a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ae5e2b881545e5bb3b90fe68b4163b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:30.753694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3065af82-7ae8-485f-888c-63972cc90a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ae5e2b881545e5bb3b90fe68b4163b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:30.754713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3065af82-7ae8-485f-888c-63972cc90a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ae5e2b881545e5bb3b90fe68b4163b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:30.764584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3065af82-7ae8-485f-888c-63972cc90a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ae5e2b881545e5bb3b90fe68b4163b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:30.773925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1172/2368] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:30 2022] DELETE /identity/v3/users/06ae5e2b881545e5bb3b90fe68b4163b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:31.077582 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:31.079927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:31.103498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e446b-75a9-4c9f-843e-4e9a41632980 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:31.200141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:31.201244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:31.201611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:31.201894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] PATH_INFO: `/v3/projects/a9e44ec35943451da81a947182c7f106` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:31.204071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e44ec35943451da81a947182c7f106)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:31.206786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:31.235372 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:31.299515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e44ec35943451da81a947182c7f106', 'request_id': 'req-03e4ccd6-b845-499c-84c2-f1f20168b6c3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:31.307263 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e44ec35943451da81a947182c7f106', 'request_id': 'req-03e4ccd6-b845-499c-84c2-f1f20168b6c3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:31.314548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e44ec35943451da81a947182c7f106', 'request_id': 'req-03e4ccd6-b845-499c-84c2-f1f20168b6c3'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:31.338911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] The token cache is being invalidate because project a9e44ec35943451da81a947182c7f106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:31.339347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:31.340173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-03e4ccd6-b845-499c-84c2-f1f20168b6c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:31.341850 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1197/2369] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:31 2022] DELETE /identity/v3/projects/a9e44ec35943451da81a947182c7f106 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:31.348201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61eeb-c623-4014-8b41-5ac6afbc9d1e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:31.348767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61eeb-c623-4014-8b41-5ac6afbc9d1e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:31.348767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61eeb-c623-4014-8b41-5ac6afbc9d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:31.374888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a7b61eeb-c623-4014-8b41-5ac6afbc9d1e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:31.376712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a7b61eeb-c623-4014-8b41-5ac6afbc9d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:31.475438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1173/2370] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:31.484279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83397df-f9ad-470f-8bfc-125e41710e0a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:31.485492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83397df-f9ad-470f-8bfc-125e41710e0a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:31.485835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83397df-f9ad-470f-8bfc-125e41710e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:31.511210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f83397df-f9ad-470f-8bfc-125e41710e0a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:31.512769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f83397df-f9ad-470f-8bfc-125e41710e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:31.522766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1198/2371] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:31.527816 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61eeb-c623-4014-8b41-5ac6afbc9d1e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:31.541419 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61eeb-c623-4014-8b41-5ac6afbc9d1e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:31.548704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf93fc-fdde-458c-a3ce-5ac7451053aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:31.549373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf93fc-fdde-458c-a3ce-5ac7451053aa admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:31.549905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf93fc-fdde-458c-a3ce-5ac7451053aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:31.550374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf93fc-fdde-458c-a3ce-5ac7451053aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:31.551143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf93fc-fdde-458c-a3ce-5ac7451053aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:15:31.551453 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf93fc-fdde-458c-a3ce-5ac7451053aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:31.553109 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf93fc-fdde-458c-a3ce-5ac7451053aa admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:31.556009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:31.570867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1174/2372] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 17:15:31.575085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83397df-f9ad-470f-8bfc-125e41710e0a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:31.606519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83397df-f9ad-470f-8bfc-125e41710e0a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:31.621960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a04b9-8aa7-4bb0-be46-07df411b8717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:31.623177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a04b9-8aa7-4bb0-be46-07df411b8717 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:31.623560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a04b9-8aa7-4bb0-be46-07df411b8717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:31.623846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a04b9-8aa7-4bb0-be46-07df411b8717 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:31.624534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a04b9-8aa7-4bb0-be46-07df411b8717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:31.626503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a04b9-8aa7-4bb0-be46-07df411b8717 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:31.664810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1199/2373] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:31 2022] POST /identity/v3/projects => generated 387 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:31.665976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf93fc-fdde-458c-a3ce-5ac7451053aa admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:31.677439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf93fc-fdde-458c-a3ce-5ac7451053aa admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:31.740597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737d926-97f1-4737-8e51-092213b9fabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:31.741150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737d926-97f1-4737-8e51-092213b9fabb admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:31.741445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737d926-97f1-4737-8e51-092213b9fabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:31.741731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737d926-97f1-4737-8e51-092213b9fabb admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:31.742381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f737d926-97f1-4737-8e51-092213b9fabb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:31.744379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f737d926-97f1-4737-8e51-092213b9fabb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:31.775758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1175/2374] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:31 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:31.777016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a04b9-8aa7-4bb0-be46-07df411b8717 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:31.788258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a04b9-8aa7-4bb0-be46-07df411b8717 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:31.793527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f0f78-e896-47c3-8741-4e01940017ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:31.794491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f0f78-e896-47c3-8741-4e01940017ce admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:31.794768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f0f78-e896-47c3-8741-4e01940017ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:31.795066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f0f78-e896-47c3-8741-4e01940017ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:31.795883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f0f78-e896-47c3-8741-4e01940017ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:31.797899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f0f78-e896-47c3-8741-4e01940017ce admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:31.809591 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1200/2375] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:31.816206 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737d926-97f1-4737-8e51-092213b9fabb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:31.829121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737d926-97f1-4737-8e51-092213b9fabb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:31.834536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cb39f-73a6-49f3-aa38-a16da4b6aa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:31.835202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cb39f-73a6-49f3-aa38-a16da4b6aa2d admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:31.835486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cb39f-73a6-49f3-aa38-a16da4b6aa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:31.835793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cb39f-73a6-49f3-aa38-a16da4b6aa2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:31.836424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cb39f-73a6-49f3-aa38-a16da4b6aa2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:31.838312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cb39f-73a6-49f3-aa38-a16da4b6aa2d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:31.854202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1176/2376] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:31.858742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f0f78-e896-47c3-8741-4e01940017ce admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:31.869216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f0f78-e896-47c3-8741-4e01940017ce admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:31.874429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088abe2-1eae-45e9-a9d5-7c294e32dad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:31.875167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088abe2-1eae-45e9-a9d5-7c294e32dad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:31.875459 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088abe2-1eae-45e9-a9d5-7c294e32dad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:31.875734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088abe2-1eae-45e9-a9d5-7c294e32dad9 admin admin] PATH_INFO: `/v3/projects/fb2dfab599f541b9b6804b8604f0134d/users/d72c19ac192c47969fcf4f68acc27d29/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:31.900692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0088abe2-1eae-45e9-a9d5-7c294e32dad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb2dfab599f541b9b6804b8604f0134d, user_id=d72c19ac192c47969fcf4f68acc27d29, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:31.906220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0088abe2-1eae-45e9-a9d5-7c294e32dad9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:31.926406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1201/2377] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:31 2022] PUT /identity/v3/projects/fb2dfab599f541b9b6804b8604f0134d/users/d72c19ac192c47969fcf4f68acc27d29/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:15:31.938260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca65f42-2b6f-4c15-b8e7-1970c964cf26 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:31.938582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca65f42-2b6f-4c15-b8e7-1970c964cf26 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:31.938809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca65f42-2b6f-4c15-b8e7-1970c964cf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:31.970294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1ca65f42-2b6f-4c15-b8e7-1970c964cf26 None None] MFA Rules not processed for user `d72c19ac192c47969fcf4f68acc27d29`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:31.971592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1ca65f42-2b6f-4c15-b8e7-1970c964cf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:32.053198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1177/2378] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:32.059870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088abe2-1eae-45e9-a9d5-7c294e32dad9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:32.160337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088abe2-1eae-45e9-a9d5-7c294e32dad9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:32.167843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd975c-698d-42e9-82e0-d1519db524ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:32.168703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd975c-698d-42e9-82e0-d1519db524ad admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:32.169174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd975c-698d-42e9-82e0-d1519db524ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:32.169708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd975c-698d-42e9-82e0-d1519db524ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:32.170569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd975c-698d-42e9-82e0-d1519db524ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:32.173421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd975c-698d-42e9-82e0-d1519db524ad admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:32.222698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1202/2379] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:32 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:15:32.227821 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca65f42-2b6f-4c15-b8e7-1970c964cf26 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:32.247694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca65f42-2b6f-4c15-b8e7-1970c964cf26 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:32.347751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b973825-e8ee-4d90-ac84-d536f8c6dc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:32.348392 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b973825-e8ee-4d90-ac84-d536f8c6dc49 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:32.348968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b973825-e8ee-4d90-ac84-d536f8c6dc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:32.348968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b973825-e8ee-4d90-ac84-d536f8c6dc49 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:32.349587 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b973825-e8ee-4d90-ac84-d536f8c6dc49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:32.352107 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b973825-e8ee-4d90-ac84-d536f8c6dc49 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:32.366586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:32.386650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1178/2380] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:32 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:15:32.388719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd975c-698d-42e9-82e0-d1519db524ad admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:32.402321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd975c-698d-42e9-82e0-d1519db524ad admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:32.409138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae1613-d47d-4fe2-bef5-87824eef0b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:32.411041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae1613-d47d-4fe2-bef5-87824eef0b1a admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:32.412287 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae1613-d47d-4fe2-bef5-87824eef0b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:32.412801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae1613-d47d-4fe2-bef5-87824eef0b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:32.413770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae1613-d47d-4fe2-bef5-87824eef0b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:32.416414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae1613-d47d-4fe2-bef5-87824eef0b1a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:32.439507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1203/2381] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:32.443829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b973825-e8ee-4d90-ac84-d536f8c6dc49 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:32.455897 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b973825-e8ee-4d90-ac84-d536f8c6dc49 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:32.460645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3a471-b162-4ace-ad25-86098ace8ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:32.461275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3a471-b162-4ace-ad25-86098ace8ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:32.461562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3a471-b162-4ace-ad25-86098ace8ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:32.461811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3a471-b162-4ace-ad25-86098ace8ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:32.462458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e3a471-b162-4ace-ad25-86098ace8ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:32.464231 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e3a471-b162-4ace-ad25-86098ace8ba0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:32.483190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1179/2382] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:32.488659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae1613-d47d-4fe2-bef5-87824eef0b1a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:32.505896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae1613-d47d-4fe2-bef5-87824eef0b1a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:32.511364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1f27a-d0cc-4e23-88ab-95b3a4b15167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:32.512057 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1f27a-d0cc-4e23-88ab-95b3a4b15167 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:32.512388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1f27a-d0cc-4e23-88ab-95b3a4b15167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:32.512670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1f27a-d0cc-4e23-88ab-95b3a4b15167 admin admin] PATH_INFO: `/v3/projects/f6348d63e72f45cda1bc8f8c35625ffc/users/253817c8c86c4990bb883bdd50580fa4/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:32.530531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:32.536344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1f27a-d0cc-4e23-88ab-95b3a4b15167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6348d63e72f45cda1bc8f8c35625ffc, user_id=253817c8c86c4990bb883bdd50580fa4, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:32.538772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1f27a-d0cc-4e23-88ab-95b3a4b15167 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:32.556974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1204/2383] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:32 2022] PUT /identity/v3/projects/f6348d63e72f45cda1bc8f8c35625ffc/users/253817c8c86c4990bb883bdd50580fa4/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:32.568153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914a760-d243-4942-ae37-5ef335173dd0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:32.568443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914a760-d243-4942-ae37-5ef335173dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:32.568736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914a760-d243-4942-ae37-5ef335173dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:32.605875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8914a760-d243-4942-ae37-5ef335173dd0 None None] MFA Rules not processed for user `253817c8c86c4990bb883bdd50580fa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:32.607517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8914a760-d243-4942-ae37-5ef335173dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:32.703316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1180/2384] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:32.708788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1f27a-d0cc-4e23-88ab-95b3a4b15167 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:32.826412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1f27a-d0cc-4e23-88ab-95b3a4b15167 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:32.832250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475510c-8a92-4ccf-b39d-6df991e1d0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:32.832922 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475510c-8a92-4ccf-b39d-6df991e1d0af admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:32.833479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475510c-8a92-4ccf-b39d-6df991e1d0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:32.833827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475510c-8a92-4ccf-b39d-6df991e1d0af admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:32.834544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475510c-8a92-4ccf-b39d-6df991e1d0af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:32.836658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475510c-8a92-4ccf-b39d-6df991e1d0af admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:32.870634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1205/2385] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:32 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:15:32.872016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914a760-d243-4942-ae37-5ef335173dd0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:32.885660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914a760-d243-4942-ae37-5ef335173dd0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:32.959567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b13ba4-c0b9-4e61-857e-fdf96be9c9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:32.960771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b13ba4-c0b9-4e61-857e-fdf96be9c9ee admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:32.961275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b13ba4-c0b9-4e61-857e-fdf96be9c9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:32.961913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b13ba4-c0b9-4e61-857e-fdf96be9c9ee admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:32.962933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b13ba4-c0b9-4e61-857e-fdf96be9c9ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:32.965505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b13ba4-c0b9-4e61-857e-fdf96be9c9ee admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:32.993605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1181/2386] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:32 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:32.998233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475510c-8a92-4ccf-b39d-6df991e1d0af admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:33.010057 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475510c-8a92-4ccf-b39d-6df991e1d0af admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:33.015148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d21e6e-21e3-454f-9786-e8c0e3d829ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:33.016422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d21e6e-21e3-454f-9786-e8c0e3d829ab admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:33.016704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d21e6e-21e3-454f-9786-e8c0e3d829ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:33.016974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d21e6e-21e3-454f-9786-e8c0e3d829ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:33.017650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d21e6e-21e3-454f-9786-e8c0e3d829ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:33.019327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d21e6e-21e3-454f-9786-e8c0e3d829ab admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:33.031125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1206/2387] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:33.036397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b13ba4-c0b9-4e61-857e-fdf96be9c9ee admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:33.048472 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b13ba4-c0b9-4e61-857e-fdf96be9c9ee admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:33.053937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae601f-fdef-41f5-a86f-185842a68c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:33.054610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae601f-fdef-41f5-a86f-185842a68c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:33.054889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae601f-fdef-41f5-a86f-185842a68c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:33.055193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae601f-fdef-41f5-a86f-185842a68c8d admin admin] PATH_INFO: `/v3/projects/ab86bceb57484601b22416b41b86a4cb/users/08407207a08f488fa08055e66fb8782b/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:33.074516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae601f-fdef-41f5-a86f-185842a68c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab86bceb57484601b22416b41b86a4cb, user_id=08407207a08f488fa08055e66fb8782b, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:33.077657 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae601f-fdef-41f5-a86f-185842a68c8d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:33.095905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1182/2388] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:33 2022] PUT /identity/v3/projects/ab86bceb57484601b22416b41b86a4cb/users/08407207a08f488fa08055e66fb8782b/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:33.100297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d21e6e-21e3-454f-9786-e8c0e3d829ab admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:33.111769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d21e6e-21e3-454f-9786-e8c0e3d829ab admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:33.179887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de09e5-7bd6-4b86-aebc-69255ac52223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:33.181028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de09e5-7bd6-4b86-aebc-69255ac52223 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:33.181028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de09e5-7bd6-4b86-aebc-69255ac52223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:33.181559 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de09e5-7bd6-4b86-aebc-69255ac52223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:33.182297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de09e5-7bd6-4b86-aebc-69255ac52223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:33.184267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de09e5-7bd6-4b86-aebc-69255ac52223 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:33.198509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1207/2389] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:33.203228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae601f-fdef-41f5-a86f-185842a68c8d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:33.219139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae601f-fdef-41f5-a86f-185842a68c8d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:33.228557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd3e0f-522d-4de4-b5f1-502ee39e36a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:33.229455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd3e0f-522d-4de4-b5f1-502ee39e36a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:33.229872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd3e0f-522d-4de4-b5f1-502ee39e36a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:33.230308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd3e0f-522d-4de4-b5f1-502ee39e36a3 admin admin] PATH_INFO: `/v3/projects/ab86bceb57484601b22416b41b86a4cb/users/08407207a08f488fa08055e66fb8782b/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:33.233809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd3e0f-522d-4de4-b5f1-502ee39e36a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab86bceb57484601b22416b41b86a4cb, user_id=08407207a08f488fa08055e66fb8782b, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:33.237478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd3e0f-522d-4de4-b5f1-502ee39e36a3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:33.254180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1183/2390] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:33 2022] PUT /identity/v3/projects/ab86bceb57484601b22416b41b86a4cb/users/08407207a08f488fa08055e66fb8782b/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:15:33.267397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b222c25-db8f-45ae-9d13-75a6aacc3336 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:33.267714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b222c25-db8f-45ae-9d13-75a6aacc3336 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:33.267874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b222c25-db8f-45ae-9d13-75a6aacc3336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:33.318702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3b222c25-db8f-45ae-9d13-75a6aacc3336 None None] MFA Rules not processed for user `08407207a08f488fa08055e66fb8782b`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:33.320321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3b222c25-db8f-45ae-9d13-75a6aacc3336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:33.387584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:33.414545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1208/2391] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:15:33.427481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd3e0f-522d-4de4-b5f1-502ee39e36a3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:33.430331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-57dd3e0f-522d-4de4-b5f1-502ee39e36a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:33.477559 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:33.518132 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd3e0f-522d-4de4-b5f1-502ee39e36a3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:33.524178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd8db8-8db5-4fba-b007-7c049927fa78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:33.525003 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd8db8-8db5-4fba-b007-7c049927fa78 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:33.525285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd8db8-8db5-4fba-b007-7c049927fa78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:33.525592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd8db8-8db5-4fba-b007-7c049927fa78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:33.542065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd8db8-8db5-4fba-b007-7c049927fa78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:33.544574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd8db8-8db5-4fba-b007-7c049927fa78 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:33.557333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1184/2392] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:34.146470 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b222c25-db8f-45ae-9d13-75a6aacc3336 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:34.149055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3b222c25-db8f-45ae-9d13-75a6aacc3336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:34.169367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b222c25-db8f-45ae-9d13-75a6aacc3336 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:34.175357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f7773-6229-41a4-b7f5-939da7bc829e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:34.176409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f7773-6229-41a4-b7f5-939da7bc829e service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:34.176921 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f7773-6229-41a4-b7f5-939da7bc829e service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:34.177434 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f7773-6229-41a4-b7f5-939da7bc829e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:34.244623 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f7773-6229-41a4-b7f5-939da7bc829e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:34.246138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f7773-6229-41a4-b7f5-939da7bc829e service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:34.257109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1209/2393] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:38.376946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd8db8-8db5-4fba-b007-7c049927fa78 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:38.385765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd8db8-8db5-4fba-b007-7c049927fa78 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:38.389594 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7539d61-f88f-4ccb-aa09-4a039214aa2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:38.390176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539d61-f88f-4ccb-aa09-4a039214aa2f service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:38.390448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539d61-f88f-4ccb-aa09-4a039214aa2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:38.391484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539d61-f88f-4ccb-aa09-4a039214aa2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:38.447654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7539d61-f88f-4ccb-aa09-4a039214aa2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:38.449072 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7539d61-f88f-4ccb-aa09-4a039214aa2f service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:38.458388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1185/2394] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:40.565801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f7773-6229-41a4-b7f5-939da7bc829e service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:40.627973 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f7773-6229-41a4-b7f5-939da7bc829e service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:40.632447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:40.633070 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:40.633420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:40.633609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] PATH_INFO: `/v3/users/d72c19ac192c47969fcf4f68acc27d29` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:40.634724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d72c19ac192c47969fcf4f68acc27d29)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:40.636691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:40.685683 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd72c19ac192c47969fcf4f68acc27d29'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:40.697851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd72c19ac192c47969fcf4f68acc27d29'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:40.705141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd72c19ac192c47969fcf4f68acc27d29'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:40.712484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd72c19ac192c47969fcf4f68acc27d29'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:40.713280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd72c19ac192c47969fcf4f68acc27d29'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:40.720418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1210/2395] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:40 2022] DELETE /identity/v3/users/d72c19ac192c47969fcf4f68acc27d29 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:40.724984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809ac2-7f2b-4ea9-b0c7-a6d4c4d8912b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:40.725449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809ac2-7f2b-4ea9-b0c7-a6d4c4d8912b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:40.725546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809ac2-7f2b-4ea9-b0c7-a6d4c4d8912b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:40.745174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-64809ac2-7f2b-4ea9-b0c7-a6d4c4d8912b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:40.746393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-64809ac2-7f2b-4ea9-b0c7-a6d4c4d8912b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:40.810081 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1186/2396] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:40.820098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:40.821540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:40.828724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:40.881857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869381a-3b8d-47ba-b7c4-85d98e88ac2a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:40.885410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39e29f-a70f-4e3c-9be4-78275d6b5ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:40.886008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39e29f-a70f-4e3c-9be4-78275d6b5ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:40.886284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39e29f-a70f-4e3c-9be4-78275d6b5ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:40.886549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39e29f-a70f-4e3c-9be4-78275d6b5ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:40.895870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39e29f-a70f-4e3c-9be4-78275d6b5ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:40.897267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39e29f-a70f-4e3c-9be4-78275d6b5ab2 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:40.907179 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1211/2397] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:41.200527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64809ac2-7f2b-4ea9-b0c7-a6d4c4d8912b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:41.202063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-64809ac2-7f2b-4ea9-b0c7-a6d4c4d8912b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:41.218135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64809ac2-7f2b-4ea9-b0c7-a6d4c4d8912b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:41.222827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:41.223559 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:41.223912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:41.224222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] PATH_INFO: `/v3/projects/fb2dfab599f541b9b6804b8604f0134d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:41.225350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb2dfab599f541b9b6804b8604f0134d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:41.227251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:41.274321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb2dfab599f541b9b6804b8604f0134d', 'request_id': 'req-c0ff3d1d-0469-4905-b377-cf516d467674'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.279743 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb2dfab599f541b9b6804b8604f0134d', 'request_id': 'req-c0ff3d1d-0469-4905-b377-cf516d467674'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.281561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:41.288098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb2dfab599f541b9b6804b8604f0134d', 'request_id': 'req-c0ff3d1d-0469-4905-b377-cf516d467674'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.300978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] The token cache is being invalidate because project fb2dfab599f541b9b6804b8604f0134d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:41.301246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.301851 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.303364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1187/2398] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:41 2022] DELETE /identity/v3/projects/fb2dfab599f541b9b6804b8604f0134d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:41.307472 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39e29f-a70f-4e3c-9be4-78275d6b5ab2 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:41.317694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39e29f-a70f-4e3c-9be4-78275d6b5ab2 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:41.373078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:41.373711 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:41.373994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:41.374271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] PATH_INFO: `/v3/users/253817c8c86c4990bb883bdd50580fa4` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:41.375914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253817c8c86c4990bb883bdd50580fa4)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:41.377870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:41.426417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253817c8c86c4990bb883bdd50580fa4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.438905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253817c8c86c4990bb883bdd50580fa4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.445804 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253817c8c86c4990bb883bdd50580fa4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.453493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253817c8c86c4990bb883bdd50580fa4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.454205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253817c8c86c4990bb883bdd50580fa4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.460363 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1212/2399] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:41 2022] DELETE /identity/v3/users/253817c8c86c4990bb883bdd50580fa4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:15:41.741711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:41.743179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:41.763059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3d1d-0469-4905-b377-cf516d467674 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:41.829441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:41.830097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:41.830373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:41.830640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] PATH_INFO: `/v3/projects/f6348d63e72f45cda1bc8f8c35625ffc` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:41.831843 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6348d63e72f45cda1bc8f8c35625ffc)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:41.833769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:41.894239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6348d63e72f45cda1bc8f8c35625ffc', 'request_id': 'req-b4a0e760-df66-4182-8a43-a19cbf9375ce'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.900707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6348d63e72f45cda1bc8f8c35625ffc', 'request_id': 'req-b4a0e760-df66-4182-8a43-a19cbf9375ce'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.910224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6348d63e72f45cda1bc8f8c35625ffc', 'request_id': 'req-b4a0e760-df66-4182-8a43-a19cbf9375ce'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.928296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] The token cache is being invalidate because project f6348d63e72f45cda1bc8f8c35625ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:41.928590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.929586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:41.931413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1188/2400] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:41 2022] DELETE /identity/v3/projects/f6348d63e72f45cda1bc8f8c35625ffc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:15:41.940447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:41.942156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:41.956014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ca4c1-159f-4298-a0b5-ae6a94a153b8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:41.983607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:42.019624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d773265-934a-497c-8f45-60d319759592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:42.020242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773265-934a-497c-8f45-60d319759592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:42.020521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773265-934a-497c-8f45-60d319759592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:42.020794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773265-934a-497c-8f45-60d319759592 admin admin] PATH_INFO: `/v3/users/08407207a08f488fa08055e66fb8782b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:42.021965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d773265-934a-497c-8f45-60d319759592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08407207a08f488fa08055e66fb8782b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:42.023940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d773265-934a-497c-8f45-60d319759592 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:42.083187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9d773265-934a-497c-8f45-60d319759592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08407207a08f488fa08055e66fb8782b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:42.097630 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9d773265-934a-497c-8f45-60d319759592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08407207a08f488fa08055e66fb8782b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:42.113793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9d773265-934a-497c-8f45-60d319759592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08407207a08f488fa08055e66fb8782b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:42.122209 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9d773265-934a-497c-8f45-60d319759592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08407207a08f488fa08055e66fb8782b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:42.122856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9d773265-934a-497c-8f45-60d319759592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08407207a08f488fa08055e66fb8782b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:42.132551 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1213/2401] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:41 2022] DELETE /identity/v3/users/08407207a08f488fa08055e66fb8782b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:42.496819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:42.498368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:42.518522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0e760-df66-4182-8a43-a19cbf9375ce admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:42.582763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:42.583419 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:42.583697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:42.583968 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] PATH_INFO: `/v3/projects/ab86bceb57484601b22416b41b86a4cb` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:42.585071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab86bceb57484601b22416b41b86a4cb)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:42.586995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:42.620622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:42.653490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab86bceb57484601b22416b41b86a4cb', 'request_id': 'req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:42.659259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab86bceb57484601b22416b41b86a4cb', 'request_id': 'req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:42.665573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab86bceb57484601b22416b41b86a4cb', 'request_id': 'req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:42.681784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] The token cache is being invalidate because project ab86bceb57484601b22416b41b86a4cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:42.682176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:42.682787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-86985bbb-7ed8-4817-8fbf-a0fcc0f933bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:42.684377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1189/2402] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:42 2022] DELETE /identity/v3/projects/ab86bceb57484601b22416b41b86a4cb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:42.690909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d22b0a-c3c9-4503-981c-7212c13b68ac None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:42.691225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d22b0a-c3c9-4503-981c-7212c13b68ac None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:42.691503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d22b0a-c3c9-4503-981c-7212c13b68ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:42.714273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d0d22b0a-c3c9-4503-981c-7212c13b68ac None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:42.715501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d0d22b0a-c3c9-4503-981c-7212c13b68ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:42.780928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1214/2403] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:42.790460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3a3a-7b45-4ae3-9d6c-dadd146e385b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:42.790997 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3a3a-7b45-4ae3-9d6c-dadd146e385b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:42.791100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3a3a-7b45-4ae3-9d6c-dadd146e385b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:42.813117 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-442c3a3a-7b45-4ae3-9d6c-dadd146e385b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:42.814372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-442c3a3a-7b45-4ae3-9d6c-dadd146e385b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:42.822418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1190/2404] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:42.827020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d22b0a-c3c9-4503-981c-7212c13b68ac None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:42.838017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d22b0a-c3c9-4503-981c-7212c13b68ac None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:42.843194 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a89192-0be3-445f-95c0-4ee79c809a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:42.843810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a89192-0be3-445f-95c0-4ee79c809a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:42.844088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a89192-0be3-445f-95c0-4ee79c809a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:42.844359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a89192-0be3-445f-95c0-4ee79c809a16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:42.845010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a89192-0be3-445f-95c0-4ee79c809a16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:15:42.845312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a89192-0be3-445f-95c0-4ee79c809a16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:42.846896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a89192-0be3-445f-95c0-4ee79c809a16 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:42.859048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1215/2405] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:15:42.864237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c3a3a-7b45-4ae3-9d6c-dadd146e385b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:42.873559 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c3a3a-7b45-4ae3-9d6c-dadd146e385b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:42.878659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73090-003d-483f-be4d-7f8e9dab5a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:42.879290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73090-003d-483f-be4d-7f8e9dab5a20 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:42.879571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73090-003d-483f-be4d-7f8e9dab5a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:42.879835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73090-003d-483f-be4d-7f8e9dab5a20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:42.880473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb73090-003d-483f-be4d-7f8e9dab5a20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:42.882340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb73090-003d-483f-be4d-7f8e9dab5a20 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:42.909853 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1191/2406] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:42 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:42.914131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a89192-0be3-445f-95c0-4ee79c809a16 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:42.924503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a89192-0be3-445f-95c0-4ee79c809a16 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:42.955677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:42.984386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0999052-83eb-47a8-8024-e1d411b0ba0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:42.985012 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0999052-83eb-47a8-8024-e1d411b0ba0e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:42.985261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0999052-83eb-47a8-8024-e1d411b0ba0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:42.985532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0999052-83eb-47a8-8024-e1d411b0ba0e admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:42.986159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0999052-83eb-47a8-8024-e1d411b0ba0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:42.988080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0999052-83eb-47a8-8024-e1d411b0ba0e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:43.009630 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1216/2407] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:42 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:43.014059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73090-003d-483f-be4d-7f8e9dab5a20 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:43.025056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73090-003d-483f-be4d-7f8e9dab5a20 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:43.029608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8624e-65b0-4e2c-8a06-5b5eb7fbcbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:43.030193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8624e-65b0-4e2c-8a06-5b5eb7fbcbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.030468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8624e-65b0-4e2c-8a06-5b5eb7fbcbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.030736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8624e-65b0-4e2c-8a06-5b5eb7fbcbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.031409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8624e-65b0-4e2c-8a06-5b5eb7fbcbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:43.033205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8624e-65b0-4e2c-8a06-5b5eb7fbcbf2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:43.043632 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1192/2408] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:43.049220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0999052-83eb-47a8-8024-e1d411b0ba0e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:43.059754 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0999052-83eb-47a8-8024-e1d411b0ba0e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:43.065087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ee4f9-9cd5-47a2-b4e8-c50720467d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:43.065685 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ee4f9-9cd5-47a2-b4e8-c50720467d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.065969 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ee4f9-9cd5-47a2-b4e8-c50720467d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.066247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ee4f9-9cd5-47a2-b4e8-c50720467d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.066886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ee4f9-9cd5-47a2-b4e8-c50720467d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:43.068522 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ee4f9-9cd5-47a2-b4e8-c50720467d3d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:43.079437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1217/2409] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:15:43.083634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8624e-65b0-4e2c-8a06-5b5eb7fbcbf2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:43.093443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8624e-65b0-4e2c-8a06-5b5eb7fbcbf2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:43.098012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff5ac8-e0bb-4ec4-b8b5-a74dd7d8e8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:43.098651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff5ac8-e0bb-4ec4-b8b5-a74dd7d8e8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.098927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff5ac8-e0bb-4ec4-b8b5-a74dd7d8e8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.099234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff5ac8-e0bb-4ec4-b8b5-a74dd7d8e8f7 admin admin] PATH_INFO: `/v3/projects/048b172509564bd386447da5857db53b/users/27bf0bfbc3be404394c4b5b9c4201aa5/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.113969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff5ac8-e0bb-4ec4-b8b5-a74dd7d8e8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=048b172509564bd386447da5857db53b, user_id=27bf0bfbc3be404394c4b5b9c4201aa5, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:43.116652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff5ac8-e0bb-4ec4-b8b5-a74dd7d8e8f7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:43.130508 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1193/2410] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:43 2022] PUT /identity/v3/projects/048b172509564bd386447da5857db53b/users/27bf0bfbc3be404394c4b5b9c4201aa5/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:43.143509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec597c-e913-428a-bdc2-e93783fa3844 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.143786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec597c-e913-428a-bdc2-e93783fa3844 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.144085 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec597c-e913-428a-bdc2-e93783fa3844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.175661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d4ec597c-e913-428a-bdc2-e93783fa3844 None None] MFA Rules not processed for user `27bf0bfbc3be404394c4b5b9c4201aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:43.176878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d4ec597c-e913-428a-bdc2-e93783fa3844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:43.247341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1218/2411] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:43.252528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff5ac8-e0bb-4ec4-b8b5-a74dd7d8e8f7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:43.316922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff5ac8-e0bb-4ec4-b8b5-a74dd7d8e8f7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:43.321802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7369800-eaaa-46c5-95ed-27dd380bb9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:43.322410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7369800-eaaa-46c5-95ed-27dd380bb9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.322692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7369800-eaaa-46c5-95ed-27dd380bb9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.322987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7369800-eaaa-46c5-95ed-27dd380bb9c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.323625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7369800-eaaa-46c5-95ed-27dd380bb9c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:43.325527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7369800-eaaa-46c5-95ed-27dd380bb9c2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:43.352741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1194/2412] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:43 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:43.356825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec597c-e913-428a-bdc2-e93783fa3844 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:43.368165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec597c-e913-428a-bdc2-e93783fa3844 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:43.426256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9277c5-9a52-40a4-a307-460a1cb98258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:43.426886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9277c5-9a52-40a4-a307-460a1cb98258 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.427198 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9277c5-9a52-40a4-a307-460a1cb98258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.427472 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9277c5-9a52-40a4-a307-460a1cb98258 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.428158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9277c5-9a52-40a4-a307-460a1cb98258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:43.430104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9277c5-9a52-40a4-a307-460a1cb98258 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:43.452785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1219/2413] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:43 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:43.456885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7369800-eaaa-46c5-95ed-27dd380bb9c2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:43.467627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7369800-eaaa-46c5-95ed-27dd380bb9c2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:43.473662 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183c4bc-3354-4c2e-9789-069c151a0422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:43.474469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183c4bc-3354-4c2e-9789-069c151a0422 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.474926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183c4bc-3354-4c2e-9789-069c151a0422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.475381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183c4bc-3354-4c2e-9789-069c151a0422 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.476368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183c4bc-3354-4c2e-9789-069c151a0422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:43.478909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183c4bc-3354-4c2e-9789-069c151a0422 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:43.496634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1195/2414] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:43.501820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9277c5-9a52-40a4-a307-460a1cb98258 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:43.514539 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9277c5-9a52-40a4-a307-460a1cb98258 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:43.521698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f6940-dcc8-4be0-bd99-0b2b303381a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:43.522569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f6940-dcc8-4be0-bd99-0b2b303381a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.522895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f6940-dcc8-4be0-bd99-0b2b303381a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.523200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f6940-dcc8-4be0-bd99-0b2b303381a3 admin admin] PATH_INFO: `/v3/projects/137f055dac9f426980432e5a9a874856/users/716e026af84d4557ba3c1534d5587a10/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.552769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f6940-dcc8-4be0-bd99-0b2b303381a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=137f055dac9f426980432e5a9a874856, user_id=716e026af84d4557ba3c1534d5587a10, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:43.555509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f6940-dcc8-4be0-bd99-0b2b303381a3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:43.570484 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1220/2415] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:43 2022] PUT /identity/v3/projects/137f055dac9f426980432e5a9a874856/users/716e026af84d4557ba3c1534d5587a10/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:43.574371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183c4bc-3354-4c2e-9789-069c151a0422 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:43.584982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183c4bc-3354-4c2e-9789-069c151a0422 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:43.602735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:43.647645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3845b1-5c34-4324-b8d1-cbfdacfd66f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:43.648350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3845b1-5c34-4324-b8d1-cbfdacfd66f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.648710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3845b1-5c34-4324-b8d1-cbfdacfd66f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.649050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3845b1-5c34-4324-b8d1-cbfdacfd66f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.649769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3845b1-5c34-4324-b8d1-cbfdacfd66f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:43.651401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3845b1-5c34-4324-b8d1-cbfdacfd66f1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:43.662310 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1196/2416] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:43.666652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f6940-dcc8-4be0-bd99-0b2b303381a3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:43.678204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f6940-dcc8-4be0-bd99-0b2b303381a3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:43.682952 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d4803-d192-4521-a8e6-4a9a769640f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:43.683633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d4803-d192-4521-a8e6-4a9a769640f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.683915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d4803-d192-4521-a8e6-4a9a769640f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.684192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d4803-d192-4521-a8e6-4a9a769640f0 admin admin] PATH_INFO: `/v3/projects/137f055dac9f426980432e5a9a874856/users/716e026af84d4557ba3c1534d5587a10/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.686208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d4803-d192-4521-a8e6-4a9a769640f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=137f055dac9f426980432e5a9a874856, user_id=716e026af84d4557ba3c1534d5587a10, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:43.688905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d4803-d192-4521-a8e6-4a9a769640f0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:43.691177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:43.701025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1221/2417] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:43 2022] PUT /identity/v3/projects/137f055dac9f426980432e5a9a874856/users/716e026af84d4557ba3c1534d5587a10/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:15:43.712778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bc0da-9859-4a09-8313-1985ec4b07e7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.712996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bc0da-9859-4a09-8313-1985ec4b07e7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.713332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bc0da-9859-4a09-8313-1985ec4b07e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.743207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1e0bc0da-9859-4a09-8313-1985ec4b07e7 None None] MFA Rules not processed for user `716e026af84d4557ba3c1534d5587a10`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:43.744435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1e0bc0da-9859-4a09-8313-1985ec4b07e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:43.817634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1197/2418] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:43 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:15:43.829387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d4803-d192-4521-a8e6-4a9a769640f0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:43.831177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-485d4803-d192-4521-a8e6-4a9a769640f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:43.895350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d4803-d192-4521-a8e6-4a9a769640f0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:43.899108 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50323c-0bcf-4450-8377-05734f3e8834 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:43.899727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50323c-0bcf-4450-8377-05734f3e8834 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:43.900014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50323c-0bcf-4450-8377-05734f3e8834 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:43.900283 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50323c-0bcf-4450-8377-05734f3e8834 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:43.910139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50323c-0bcf-4450-8377-05734f3e8834 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:43.911610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50323c-0bcf-4450-8377-05734f3e8834 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:43.921273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1222/2419] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:43 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:44.512738 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bc0da-9859-4a09-8313-1985ec4b07e7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:44.591299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bc0da-9859-4a09-8313-1985ec4b07e7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:44.595796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:44.596437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:44.596710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:44.596980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] PATH_INFO: `/v3/users/27bf0bfbc3be404394c4b5b9c4201aa5` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:44.598131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27bf0bfbc3be404394c4b5b9c4201aa5)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:44.600082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:44.648638 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:44.651567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27bf0bfbc3be404394c4b5b9c4201aa5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:44.673926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27bf0bfbc3be404394c4b5b9c4201aa5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:44.683008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27bf0bfbc3be404394c4b5b9c4201aa5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:44.683746 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27bf0bfbc3be404394c4b5b9c4201aa5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:44.694064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27bf0bfbc3be404394c4b5b9c4201aa5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:44.700890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1198/2420] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:44 2022] DELETE /identity/v3/users/27bf0bfbc3be404394c4b5b9c4201aa5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:44.705667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e12352-58cb-4f1c-8102-7af39cd9993d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:44.706206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e12352-58cb-4f1c-8102-7af39cd9993d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:44.706206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e12352-58cb-4f1c-8102-7af39cd9993d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:44.725933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-35e12352-58cb-4f1c-8102-7af39cd9993d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:44.727184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-35e12352-58cb-4f1c-8102-7af39cd9993d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:44.811716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1223/2421] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:44.822649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:44.824218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:44.885449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a8fd9-b106-4ca6-82c7-6fb7775369a9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:44.889507 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf735668-571e-4ca3-a619-e9e9752743be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:44.890127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf735668-571e-4ca3-a619-e9e9752743be service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:44.890433 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf735668-571e-4ca3-a619-e9e9752743be service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:44.890706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf735668-571e-4ca3-a619-e9e9752743be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:44.899844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf735668-571e-4ca3-a619-e9e9752743be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:44.901270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf735668-571e-4ca3-a619-e9e9752743be service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:44.911223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1199/2422] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:45.188672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e12352-58cb-4f1c-8102-7af39cd9993d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:45.190200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-35e12352-58cb-4f1c-8102-7af39cd9993d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:45.205648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e12352-58cb-4f1c-8102-7af39cd9993d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:45.213336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:45.215645 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:45.216016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:45.216589 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] PATH_INFO: `/v3/projects/048b172509564bd386447da5857db53b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:45.218838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] RBAC: Authorizing `identity:delete_project(project_id=048b172509564bd386447da5857db53b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:45.220903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:45.272924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048b172509564bd386447da5857db53b', 'request_id': 'req-e48788ea-bd0d-4510-bdbc-9cc110505ecd'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.279174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048b172509564bd386447da5857db53b', 'request_id': 'req-e48788ea-bd0d-4510-bdbc-9cc110505ecd'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.286148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048b172509564bd386447da5857db53b', 'request_id': 'req-e48788ea-bd0d-4510-bdbc-9cc110505ecd'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.299733 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] The token cache is being invalidate because project 048b172509564bd386447da5857db53b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:45.300032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.300678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.302194 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1224/2423] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:45 2022] DELETE /identity/v3/projects/048b172509564bd386447da5857db53b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:45.306845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf735668-571e-4ca3-a619-e9e9752743be service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:45.317105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf735668-571e-4ca3-a619-e9e9752743be service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:45.375950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4dc02-feee-463a-a943-f11eb4fb916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:45.376489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4dc02-feee-463a-a943-f11eb4fb916a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:45.376769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4dc02-feee-463a-a943-f11eb4fb916a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:45.377044 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4dc02-feee-463a-a943-f11eb4fb916a admin admin] PATH_INFO: `/v3/users/716e026af84d4557ba3c1534d5587a10` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:45.378164 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4dc02-feee-463a-a943-f11eb4fb916a admin admin] RBAC: Authorizing `identity:delete_user(user_id=716e026af84d4557ba3c1534d5587a10)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:45.380226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4dc02-feee-463a-a943-f11eb4fb916a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:45.430664 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f0a4dc02-feee-463a-a943-f11eb4fb916a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '716e026af84d4557ba3c1534d5587a10'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.443019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f0a4dc02-feee-463a-a943-f11eb4fb916a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '716e026af84d4557ba3c1534d5587a10'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.444234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:45.452375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f0a4dc02-feee-463a-a943-f11eb4fb916a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '716e026af84d4557ba3c1534d5587a10'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.454275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f0a4dc02-feee-463a-a943-f11eb4fb916a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '716e026af84d4557ba3c1534d5587a10'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.465108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f0a4dc02-feee-463a-a943-f11eb4fb916a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '716e026af84d4557ba3c1534d5587a10'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.472777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1200/2424] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:45 2022] DELETE /identity/v3/users/716e026af84d4557ba3c1534d5587a10 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:15:45.753835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:45.753835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:45.766638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48788ea-bd0d-4510-bdbc-9cc110505ecd admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:45.783681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:45.826425 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:45.827091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:45.827370 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:45.827640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] PATH_INFO: `/v3/projects/137f055dac9f426980432e5a9a874856` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:45.829308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=137f055dac9f426980432e5a9a874856)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:45.831233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:45.880768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '137f055dac9f426980432e5a9a874856', 'request_id': 'req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.887202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '137f055dac9f426980432e5a9a874856', 'request_id': 'req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.893507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '137f055dac9f426980432e5a9a874856', 'request_id': 'req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.907619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] The token cache is being invalidate because project 137f055dac9f426980432e5a9a874856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:45.907923 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.908579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-db1d6381-a4e7-4f5b-bca8-02f2d1a669f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:45.910687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1225/2425] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:45 2022] DELETE /identity/v3/projects/137f055dac9f426980432e5a9a874856 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:45.915743 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917cbe4-9958-40a2-9a92-c87c893b833a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:45.916252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917cbe4-9958-40a2-9a92-c87c893b833a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:45.916348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917cbe4-9958-40a2-9a92-c87c893b833a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:45.936473 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5917cbe4-9958-40a2-9a92-c87c893b833a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:45.938228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5917cbe4-9958-40a2-9a92-c87c893b833a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:46.005354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1201/2426] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:46.013265 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f35291-8f9d-46cd-a5da-d6797c7c9bdb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.013549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f35291-8f9d-46cd-a5da-d6797c7c9bdb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.013826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f35291-8f9d-46cd-a5da-d6797c7c9bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.039909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-02f35291-8f9d-46cd-a5da-d6797c7c9bdb None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:46.041147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-02f35291-8f9d-46cd-a5da-d6797c7c9bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:46.050496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1226/2427] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:46.054970 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917cbe4-9958-40a2-9a92-c87c893b833a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:46.065628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917cbe4-9958-40a2-9a92-c87c893b833a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:46.070497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef11471-f913-4f69-88a2-4f250c65d364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:46.071147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef11471-f913-4f69-88a2-4f250c65d364 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.071428 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef11471-f913-4f69-88a2-4f250c65d364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.071702 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef11471-f913-4f69-88a2-4f250c65d364 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.072356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef11471-f913-4f69-88a2-4f250c65d364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:15:46.072667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef11471-f913-4f69-88a2-4f250c65d364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:46.074278 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef11471-f913-4f69-88a2-4f250c65d364 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:46.086327 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1202/2428] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:15:46.090849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f35291-8f9d-46cd-a5da-d6797c7c9bdb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:46.101786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f35291-8f9d-46cd-a5da-d6797c7c9bdb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:46.112730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6159f41-f9f8-476d-a06c-612c4094c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:46.113351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6159f41-f9f8-476d-a06c-612c4094c905 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.113634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6159f41-f9f8-476d-a06c-612c4094c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.113911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6159f41-f9f8-476d-a06c-612c4094c905 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.114570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6159f41-f9f8-476d-a06c-612c4094c905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:46.116662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6159f41-f9f8-476d-a06c-612c4094c905 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:46.149436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1227/2429] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:46 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:46.153214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef11471-f913-4f69-88a2-4f250c65d364 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:46.163502 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef11471-f913-4f69-88a2-4f250c65d364 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:46.224549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aecce7-7bcb-4449-9a08-2cc06daec950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:46.224549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aecce7-7bcb-4449-9a08-2cc06daec950 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.224549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aecce7-7bcb-4449-9a08-2cc06daec950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.224549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aecce7-7bcb-4449-9a08-2cc06daec950 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.228072 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aecce7-7bcb-4449-9a08-2cc06daec950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:46.228072 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aecce7-7bcb-4449-9a08-2cc06daec950 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:46.248887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1203/2430] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:46 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:46.253974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6159f41-f9f8-476d-a06c-612c4094c905 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:46.265529 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6159f41-f9f8-476d-a06c-612c4094c905 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:46.270610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938755ab-aebd-46b2-9422-812a6263b6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:46.271113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938755ab-aebd-46b2-9422-812a6263b6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.271394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938755ab-aebd-46b2-9422-812a6263b6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.271709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938755ab-aebd-46b2-9422-812a6263b6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.272353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938755ab-aebd-46b2-9422-812a6263b6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:46.273931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938755ab-aebd-46b2-9422-812a6263b6f9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:46.284868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1228/2431] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:46.289320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aecce7-7bcb-4449-9a08-2cc06daec950 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:46.300649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aecce7-7bcb-4449-9a08-2cc06daec950 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:46.305423 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e0ea9-c7bc-4588-857b-f707d63d82d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:46.305980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e0ea9-c7bc-4588-857b-f707d63d82d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.306260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e0ea9-c7bc-4588-857b-f707d63d82d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.306531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e0ea9-c7bc-4588-857b-f707d63d82d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.307199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e0ea9-c7bc-4588-857b-f707d63d82d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:46.308799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e0ea9-c7bc-4588-857b-f707d63d82d0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:46.324514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1204/2432] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:46.328998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938755ab-aebd-46b2-9422-812a6263b6f9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:46.339292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938755ab-aebd-46b2-9422-812a6263b6f9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:46.343962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc644ca3-3ad4-4300-afd2-0c952359ffd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:46.344599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc644ca3-3ad4-4300-afd2-0c952359ffd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.344875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc644ca3-3ad4-4300-afd2-0c952359ffd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.345145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc644ca3-3ad4-4300-afd2-0c952359ffd9 admin admin] PATH_INFO: `/v3/projects/dde67338617a4012a7db2f196eefe9aa/users/3b52d67e1c7d47e6bee1847be39d3423/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.361613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc644ca3-3ad4-4300-afd2-0c952359ffd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde67338617a4012a7db2f196eefe9aa, user_id=3b52d67e1c7d47e6bee1847be39d3423, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:46.364271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc644ca3-3ad4-4300-afd2-0c952359ffd9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:46.378170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1229/2433] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:46 2022] PUT /identity/v3/projects/dde67338617a4012a7db2f196eefe9aa/users/3b52d67e1c7d47e6bee1847be39d3423/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:46.389480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864833-15eb-420b-b0dd-2f8e4084ecc0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.389766 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864833-15eb-420b-b0dd-2f8e4084ecc0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.390070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864833-15eb-420b-b0dd-2f8e4084ecc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.420614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-16864833-15eb-420b-b0dd-2f8e4084ecc0 None None] MFA Rules not processed for user `3b52d67e1c7d47e6bee1847be39d3423`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:46.422731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-16864833-15eb-420b-b0dd-2f8e4084ecc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:46.465459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:46.498819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1205/2434] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:46.504380 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc644ca3-3ad4-4300-afd2-0c952359ffd9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:46.563405 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:46.581700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc644ca3-3ad4-4300-afd2-0c952359ffd9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:46.587105 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b186a88-5505-4202-9bf4-ba5d0b949cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:46.587888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b186a88-5505-4202-9bf4-ba5d0b949cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.588183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b186a88-5505-4202-9bf4-ba5d0b949cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.588457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b186a88-5505-4202-9bf4-ba5d0b949cd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.589108 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b186a88-5505-4202-9bf4-ba5d0b949cd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:46.591060 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b186a88-5505-4202-9bf4-ba5d0b949cd6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:46.621367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1230/2435] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:46 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:46.636772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864833-15eb-420b-b0dd-2f8e4084ecc0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:46.648168 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864833-15eb-420b-b0dd-2f8e4084ecc0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:46.721992 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bde5e-6ba6-45dc-a920-2dec57bc4053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:46.722698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bde5e-6ba6-45dc-a920-2dec57bc4053 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.723331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bde5e-6ba6-45dc-a920-2dec57bc4053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.723717 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bde5e-6ba6-45dc-a920-2dec57bc4053 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.724425 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bde5e-6ba6-45dc-a920-2dec57bc4053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:46.726528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bde5e-6ba6-45dc-a920-2dec57bc4053 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:46.757656 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1206/2436] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:46 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:46.759535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b186a88-5505-4202-9bf4-ba5d0b949cd6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:46.774471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b186a88-5505-4202-9bf4-ba5d0b949cd6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:46.780365 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fda9f-417b-40c8-ad47-23e846450687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:46.781135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fda9f-417b-40c8-ad47-23e846450687 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.781503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fda9f-417b-40c8-ad47-23e846450687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.782359 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fda9f-417b-40c8-ad47-23e846450687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.782478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fda9f-417b-40c8-ad47-23e846450687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:46.784314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fda9f-417b-40c8-ad47-23e846450687 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:46.799396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1231/2437] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:46.804038 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bde5e-6ba6-45dc-a920-2dec57bc4053 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:46.817018 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bde5e-6ba6-45dc-a920-2dec57bc4053 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:46.821860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c61d07-f6b8-4396-b5bb-e6a20cbc6374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:46.822466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c61d07-f6b8-4396-b5bb-e6a20cbc6374 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.822747 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c61d07-f6b8-4396-b5bb-e6a20cbc6374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.823105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c61d07-f6b8-4396-b5bb-e6a20cbc6374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.823763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c61d07-f6b8-4396-b5bb-e6a20cbc6374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:46.825561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c61d07-f6b8-4396-b5bb-e6a20cbc6374 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:46.838334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1207/2438] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:46.842571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fda9f-417b-40c8-ad47-23e846450687 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:46.852843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fda9f-417b-40c8-ad47-23e846450687 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:46.858153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947883b-2f40-4f7d-9fe0-a9c5842691ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:46.859350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947883b-2f40-4f7d-9fe0-a9c5842691ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.859648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947883b-2f40-4f7d-9fe0-a9c5842691ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.859915 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947883b-2f40-4f7d-9fe0-a9c5842691ed admin admin] PATH_INFO: `/v3/projects/399f9ae02b05483f9295439e82d1cd8e/users/a0e551f8389e47aebbbf6ae5ecc659a6/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.876682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947883b-2f40-4f7d-9fe0-a9c5842691ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=399f9ae02b05483f9295439e82d1cd8e, user_id=a0e551f8389e47aebbbf6ae5ecc659a6, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:46.879335 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947883b-2f40-4f7d-9fe0-a9c5842691ed admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:46.893299 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1232/2439] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:46 2022] PUT /identity/v3/projects/399f9ae02b05483f9295439e82d1cd8e/users/a0e551f8389e47aebbbf6ae5ecc659a6/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:46.903622 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e5509-aec4-486b-a936-d4eb57fb6608 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:46.903917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e5509-aec4-486b-a936-d4eb57fb6608 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:46.904197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e5509-aec4-486b-a936-d4eb57fb6608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:46.933758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ec9e5509-aec4-486b-a936-d4eb57fb6608 None None] MFA Rules not processed for user `a0e551f8389e47aebbbf6ae5ecc659a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:46.935004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ec9e5509-aec4-486b-a936-d4eb57fb6608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:47.003840 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1208/2440] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:47.008837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947883b-2f40-4f7d-9fe0-a9c5842691ed admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:47.081153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947883b-2f40-4f7d-9fe0-a9c5842691ed admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:47.086811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246bcd3-0eb6-4109-b133-56edd0181e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:47.087420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246bcd3-0eb6-4109-b133-56edd0181e22 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:47.087696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246bcd3-0eb6-4109-b133-56edd0181e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:47.087966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246bcd3-0eb6-4109-b133-56edd0181e22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:47.088632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246bcd3-0eb6-4109-b133-56edd0181e22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:47.090562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246bcd3-0eb6-4109-b133-56edd0181e22 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:47.121531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1233/2441] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:47 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:47.126131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e5509-aec4-486b-a936-d4eb57fb6608 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:47.137308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e5509-aec4-486b-a936-d4eb57fb6608 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:47.203614 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:47.221292 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e7a4d-87ca-4747-8d88-986e37119dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:47.222101 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e7a4d-87ca-4747-8d88-986e37119dec admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:47.222550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e7a4d-87ca-4747-8d88-986e37119dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:47.222830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e7a4d-87ca-4747-8d88-986e37119dec admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:47.223517 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e7a4d-87ca-4747-8d88-986e37119dec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:47.225589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e7a4d-87ca-4747-8d88-986e37119dec admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:47.248938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1209/2442] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:47 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:47.253461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246bcd3-0eb6-4109-b133-56edd0181e22 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:47.264295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246bcd3-0eb6-4109-b133-56edd0181e22 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:47.270067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa8db8-d104-4510-8b55-c097c63b1e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:47.270694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa8db8-d104-4510-8b55-c097c63b1e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:47.271000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa8db8-d104-4510-8b55-c097c63b1e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:47.271280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa8db8-d104-4510-8b55-c097c63b1e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:47.271933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa8db8-d104-4510-8b55-c097c63b1e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:47.273530 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa8db8-d104-4510-8b55-c097c63b1e4b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:47.285197 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1234/2443] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:47.289860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e7a4d-87ca-4747-8d88-986e37119dec admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:47.300705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e7a4d-87ca-4747-8d88-986e37119dec admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:47.305585 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f4dcb-4311-4956-b75a-91821a19e2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:47.306248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f4dcb-4311-4956-b75a-91821a19e2d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:47.306530 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f4dcb-4311-4956-b75a-91821a19e2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:47.306803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f4dcb-4311-4956-b75a-91821a19e2d0 admin admin] PATH_INFO: `/v3/projects/aefb7cabcf164067bcd25b38b2fe8b79/users/77adc5e54f1b4dadb2cf0fc29ff3d70f/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:47.323636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f4dcb-4311-4956-b75a-91821a19e2d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aefb7cabcf164067bcd25b38b2fe8b79, user_id=77adc5e54f1b4dadb2cf0fc29ff3d70f, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:47.326295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f4dcb-4311-4956-b75a-91821a19e2d0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:47.340324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1210/2444] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:47 2022] PUT /identity/v3/projects/aefb7cabcf164067bcd25b38b2fe8b79/users/77adc5e54f1b4dadb2cf0fc29ff3d70f/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:47.344327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa8db8-d104-4510-8b55-c097c63b1e4b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:47.354277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa8db8-d104-4510-8b55-c097c63b1e4b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:47.412253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e66ca-f6fe-4606-a85e-0d3d2b008962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:47.412864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e66ca-f6fe-4606-a85e-0d3d2b008962 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:47.413140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e66ca-f6fe-4606-a85e-0d3d2b008962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:47.413407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e66ca-f6fe-4606-a85e-0d3d2b008962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:47.414049 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e66ca-f6fe-4606-a85e-0d3d2b008962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:47.415661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e66ca-f6fe-4606-a85e-0d3d2b008962 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:47.426097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1235/2445] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:47.430536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f4dcb-4311-4956-b75a-91821a19e2d0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:47.441727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f4dcb-4311-4956-b75a-91821a19e2d0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:47.446572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b87a88-1c63-4f3f-8b04-866e184fa6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:47.447268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b87a88-1c63-4f3f-8b04-866e184fa6b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:47.447579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b87a88-1c63-4f3f-8b04-866e184fa6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:47.447859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b87a88-1c63-4f3f-8b04-866e184fa6b0 admin admin] PATH_INFO: `/v3/projects/aefb7cabcf164067bcd25b38b2fe8b79/users/77adc5e54f1b4dadb2cf0fc29ff3d70f/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:47.449792 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b87a88-1c63-4f3f-8b04-866e184fa6b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aefb7cabcf164067bcd25b38b2fe8b79, user_id=77adc5e54f1b4dadb2cf0fc29ff3d70f, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:47.452465 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b87a88-1c63-4f3f-8b04-866e184fa6b0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:47.464150 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1211/2446] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:47 2022] PUT /identity/v3/projects/aefb7cabcf164067bcd25b38b2fe8b79/users/77adc5e54f1b4dadb2cf0fc29ff3d70f/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:47.474665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75250a5b-9aa2-4687-a83b-4e7e0f992b7b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:47.474975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75250a5b-9aa2-4687-a83b-4e7e0f992b7b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:47.475290 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75250a5b-9aa2-4687-a83b-4e7e0f992b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:47.489121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:47.507177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-75250a5b-9aa2-4687-a83b-4e7e0f992b7b None None] MFA Rules not processed for user `77adc5e54f1b4dadb2cf0fc29ff3d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:47.508444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-75250a5b-9aa2-4687-a83b-4e7e0f992b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:47.590300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1236/2447] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:47.602039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b87a88-1c63-4f3f-8b04-866e184fa6b0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:47.603777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c4b87a88-1c63-4f3f-8b04-866e184fa6b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:47.664848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b87a88-1c63-4f3f-8b04-866e184fa6b0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:47.668542 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e4e0e-abcc-4065-bbec-f64fcac018ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:47.669185 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e4e0e-abcc-4065-bbec-f64fcac018ea service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:47.669463 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e4e0e-abcc-4065-bbec-f64fcac018ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:47.669733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e4e0e-abcc-4065-bbec-f64fcac018ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:47.727921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e4e0e-abcc-4065-bbec-f64fcac018ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:47.729314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e4e0e-abcc-4065-bbec-f64fcac018ea service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:47.739158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1212/2448] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:49.279253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75250a5b-9aa2-4687-a83b-4e7e0f992b7b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:49.280677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-75250a5b-9aa2-4687-a83b-4e7e0f992b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:49.292784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75250a5b-9aa2-4687-a83b-4e7e0f992b7b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:49.296849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e37bf-52f2-4217-a15a-c25b89674539 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:49.297725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e37bf-52f2-4217-a15a-c25b89674539 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:49.298000 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e37bf-52f2-4217-a15a-c25b89674539 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:49.298262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e37bf-52f2-4217-a15a-c25b89674539 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:49.362513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e37bf-52f2-4217-a15a-c25b89674539 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:49.363908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e37bf-52f2-4217-a15a-c25b89674539 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:49.374202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1237/2449] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:50.053640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e4e0e-abcc-4065-bbec-f64fcac018ea service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:50.062870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e4e0e-abcc-4065-bbec-f64fcac018ea service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:50.066940 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b90348-b34a-4648-ac54-243fa8041e45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:50.067583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b90348-b34a-4648-ac54-243fa8041e45 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:50.067860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b90348-b34a-4648-ac54-243fa8041e45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:50.068131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b90348-b34a-4648-ac54-243fa8041e45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:50.077988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b90348-b34a-4648-ac54-243fa8041e45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:50.079800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b90348-b34a-4648-ac54-243fa8041e45 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:50.090485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1213/2450] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:15:54.117856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e37bf-52f2-4217-a15a-c25b89674539 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:54.179609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e37bf-52f2-4217-a15a-c25b89674539 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:54.184088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:54.184698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:54.184974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:54.185244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] PATH_INFO: `/v3/users/3b52d67e1c7d47e6bee1847be39d3423` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:54.186338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b52d67e1c7d47e6bee1847be39d3423)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:54.188258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:54.222959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:54.236807 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b52d67e1c7d47e6bee1847be39d3423'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:54.248885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b52d67e1c7d47e6bee1847be39d3423'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:54.255843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b52d67e1c7d47e6bee1847be39d3423'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:54.263991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b52d67e1c7d47e6bee1847be39d3423'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:54.264724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b52d67e1c7d47e6bee1847be39d3423'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:54.271540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1238/2451] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:54 2022] DELETE /identity/v3/users/3b52d67e1c7d47e6bee1847be39d3423 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:54.276093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4352e-9999-48c4-904b-be7ba3cc23e9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:54.276575 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4352e-9999-48c4-904b-be7ba3cc23e9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:54.276673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4352e-9999-48c4-904b-be7ba3cc23e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:54.296461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-63f4352e-9999-48c4-904b-be7ba3cc23e9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:54.297930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-63f4352e-9999-48c4-904b-be7ba3cc23e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:54.332114 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:54.358316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1214/2452] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:54.368193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:54.369628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:54.428731 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3fdbc-9a27-49c0-afae-8c347f7a39ad admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:54.432398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633a08-f2ee-4685-93b0-70bcc46738ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:54.432998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633a08-f2ee-4685-93b0-70bcc46738ee service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:54.433273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633a08-f2ee-4685-93b0-70bcc46738ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:54.433540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633a08-f2ee-4685-93b0-70bcc46738ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:54.443054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97633a08-f2ee-4685-93b0-70bcc46738ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:54.444457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97633a08-f2ee-4685-93b0-70bcc46738ee service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:54.454617 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1239/2453] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:54.751860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4352e-9999-48c4-904b-be7ba3cc23e9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:54.754544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-63f4352e-9999-48c4-904b-be7ba3cc23e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:54.782072 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4352e-9999-48c4-904b-be7ba3cc23e9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:54.789011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:54.789660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:54.789933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:54.790216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] PATH_INFO: `/v3/projects/dde67338617a4012a7db2f196eefe9aa` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:54.791381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dde67338617a4012a7db2f196eefe9aa)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:54.793336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:54.846827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dde67338617a4012a7db2f196eefe9aa', 'request_id': 'req-7955c383-a19c-4cfe-b007-1e0d26a7af66'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:54.852892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dde67338617a4012a7db2f196eefe9aa', 'request_id': 'req-7955c383-a19c-4cfe-b007-1e0d26a7af66'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:54.859499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dde67338617a4012a7db2f196eefe9aa', 'request_id': 'req-7955c383-a19c-4cfe-b007-1e0d26a7af66'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:54.872123 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] The token cache is being invalidate because project dde67338617a4012a7db2f196eefe9aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:54.872435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:54.873060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:54.875129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1215/2454] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:54 2022] DELETE /identity/v3/projects/dde67338617a4012a7db2f196eefe9aa => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:54.879251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633a08-f2ee-4685-93b0-70bcc46738ee service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:54.889141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633a08-f2ee-4685-93b0-70bcc46738ee service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:54.947189 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:54.948360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:54.948903 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:54.949403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] PATH_INFO: `/v3/users/a0e551f8389e47aebbbf6ae5ecc659a6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:54.950991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e551f8389e47aebbbf6ae5ecc659a6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:54.954096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:55.000901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e551f8389e47aebbbf6ae5ecc659a6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.006445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:55.016974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e551f8389e47aebbbf6ae5ecc659a6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.024550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e551f8389e47aebbbf6ae5ecc659a6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.031548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e551f8389e47aebbbf6ae5ecc659a6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.032166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e551f8389e47aebbbf6ae5ecc659a6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.038059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1240/2455] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:54 2022] DELETE /identity/v3/users/a0e551f8389e47aebbbf6ae5ecc659a6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:55.330887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:55.332413 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:55.345760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955c383-a19c-4cfe-b007-1e0d26a7af66 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:55.406828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:55.407481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:55.407758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:55.408026 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] PATH_INFO: `/v3/projects/399f9ae02b05483f9295439e82d1cd8e` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:55.409129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=399f9ae02b05483f9295439e82d1cd8e)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:55.411043 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:55.457134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399f9ae02b05483f9295439e82d1cd8e', 'request_id': 'req-8bcdb217-3e15-41a8-8911-7c9d20055263'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.462644 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399f9ae02b05483f9295439e82d1cd8e', 'request_id': 'req-8bcdb217-3e15-41a8-8911-7c9d20055263'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.469577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399f9ae02b05483f9295439e82d1cd8e', 'request_id': 'req-8bcdb217-3e15-41a8-8911-7c9d20055263'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.472135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:55.484552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] The token cache is being invalidate because project 399f9ae02b05483f9295439e82d1cd8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:55.484846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.485487 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.487000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1216/2456] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:55 2022] DELETE /identity/v3/projects/399f9ae02b05483f9295439e82d1cd8e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:55.491075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:55.492385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:55.505079 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf424a-0ed9-4757-92db-151d4f3e5e20 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:55.563925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d91f78-f2ac-4dfd-bf68-c8f65fe5af11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:55.564556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d91f78-f2ac-4dfd-bf68-c8f65fe5af11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:55.564838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d91f78-f2ac-4dfd-bf68-c8f65fe5af11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:55.565112 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d91f78-f2ac-4dfd-bf68-c8f65fe5af11 admin admin] PATH_INFO: `/v3/users/77adc5e54f1b4dadb2cf0fc29ff3d70f` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:55.566226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d91f78-f2ac-4dfd-bf68-c8f65fe5af11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77adc5e54f1b4dadb2cf0fc29ff3d70f)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:55.568174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d91f78-f2ac-4dfd-bf68-c8f65fe5af11 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:55.620017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-87d91f78-f2ac-4dfd-bf68-c8f65fe5af11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77adc5e54f1b4dadb2cf0fc29ff3d70f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.634810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-87d91f78-f2ac-4dfd-bf68-c8f65fe5af11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77adc5e54f1b4dadb2cf0fc29ff3d70f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.642234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-87d91f78-f2ac-4dfd-bf68-c8f65fe5af11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77adc5e54f1b4dadb2cf0fc29ff3d70f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.649774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-87d91f78-f2ac-4dfd-bf68-c8f65fe5af11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77adc5e54f1b4dadb2cf0fc29ff3d70f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.650709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-87d91f78-f2ac-4dfd-bf68-c8f65fe5af11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77adc5e54f1b4dadb2cf0fc29ff3d70f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:55.657088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1241/2457] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:55 2022] DELETE /identity/v3/users/77adc5e54f1b4dadb2cf0fc29ff3d70f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:55.939719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:55.941170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:55.955192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdb217-3e15-41a8-8911-7c9d20055263 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:56.018574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:56.019308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.019666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.019863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] PATH_INFO: `/v3/projects/aefb7cabcf164067bcd25b38b2fe8b79` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.021234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aefb7cabcf164067bcd25b38b2fe8b79)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:56.023462 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:56.074924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aefb7cabcf164067bcd25b38b2fe8b79', 'request_id': 'req-89db8e37-2394-455e-802a-a127df74b66e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:56.081463 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aefb7cabcf164067bcd25b38b2fe8b79', 'request_id': 'req-89db8e37-2394-455e-802a-a127df74b66e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:56.088610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aefb7cabcf164067bcd25b38b2fe8b79', 'request_id': 'req-89db8e37-2394-455e-802a-a127df74b66e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:56.103863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] The token cache is being invalidate because project aefb7cabcf164067bcd25b38b2fe8b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:56.104151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:56.104912 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89db8e37-2394-455e-802a-a127df74b66e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:56.106402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1217/2458] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:55 2022] DELETE /identity/v3/projects/aefb7cabcf164067bcd25b38b2fe8b79 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:56.112345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0ef47-95c5-4b80-ae7d-66ffd81f202e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.112640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0ef47-95c5-4b80-ae7d-66ffd81f202e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.112912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0ef47-95c5-4b80-ae7d-66ffd81f202e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.134998 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b1f0ef47-95c5-4b80-ae7d-66ffd81f202e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:56.136336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b1f0ef47-95c5-4b80-ae7d-66ffd81f202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:56.190844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:56.208029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1242/2459] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:56.274312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342137dd-6c1a-49f0-9191-2ee71b0848d8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.274719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342137dd-6c1a-49f0-9191-2ee71b0848d8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.274878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342137dd-6c1a-49f0-9191-2ee71b0848d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.295552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-342137dd-6c1a-49f0-9191-2ee71b0848d8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:56.296770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-342137dd-6c1a-49f0-9191-2ee71b0848d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:56.304863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1218/2460] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:56.309276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0ef47-95c5-4b80-ae7d-66ffd81f202e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:56.319652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0ef47-95c5-4b80-ae7d-66ffd81f202e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:56.324217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec887a-4bab-4a8b-aa5f-5070a1478e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:56.324808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec887a-4bab-4a8b-aa5f-5070a1478e8d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.325090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec887a-4bab-4a8b-aa5f-5070a1478e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.325361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec887a-4bab-4a8b-aa5f-5070a1478e8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.325994 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec887a-4bab-4a8b-aa5f-5070a1478e8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:15:56.326293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec887a-4bab-4a8b-aa5f-5070a1478e8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:56.328891 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec887a-4bab-4a8b-aa5f-5070a1478e8d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:56.343138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1243/2461] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:15:56.347859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342137dd-6c1a-49f0-9191-2ee71b0848d8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:56.357091 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342137dd-6c1a-49f0-9191-2ee71b0848d8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:56.361799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed92e3-1b22-4b5b-a458-e70669d25a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:56.362395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed92e3-1b22-4b5b-a458-e70669d25a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.363066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed92e3-1b22-4b5b-a458-e70669d25a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.363066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed92e3-1b22-4b5b-a458-e70669d25a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.363589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed92e3-1b22-4b5b-a458-e70669d25a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:56.365486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed92e3-1b22-4b5b-a458-e70669d25a29 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:56.399002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1219/2462] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:56 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:56.403417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec887a-4bab-4a8b-aa5f-5070a1478e8d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:56.414171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec887a-4bab-4a8b-aa5f-5070a1478e8d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:56.471050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a905de-a274-4858-8e0e-31f3bd43f854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:56.471785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a905de-a274-4858-8e0e-31f3bd43f854 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.471933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a905de-a274-4858-8e0e-31f3bd43f854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.472206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a905de-a274-4858-8e0e-31f3bd43f854 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.472845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a905de-a274-4858-8e0e-31f3bd43f854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:56.474744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a905de-a274-4858-8e0e-31f3bd43f854 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:56.495973 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1244/2463] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:56 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:15:56.500630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed92e3-1b22-4b5b-a458-e70669d25a29 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:56.512143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed92e3-1b22-4b5b-a458-e70669d25a29 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:56.517737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044503b-6f43-4e45-b2e0-0c5689e4f58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:56.518528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044503b-6f43-4e45-b2e0-0c5689e4f58b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.519000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044503b-6f43-4e45-b2e0-0c5689e4f58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.519329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044503b-6f43-4e45-b2e0-0c5689e4f58b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.520201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044503b-6f43-4e45-b2e0-0c5689e4f58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:56.522270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044503b-6f43-4e45-b2e0-0c5689e4f58b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:56.533031 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1220/2464] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:56.537129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a905de-a274-4858-8e0e-31f3bd43f854 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:56.548233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a905de-a274-4858-8e0e-31f3bd43f854 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:56.552852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f765f-b5e4-4c5c-8706-d83184ff6d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:56.553446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f765f-b5e4-4c5c-8706-d83184ff6d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.553728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f765f-b5e4-4c5c-8706-d83184ff6d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.554001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f765f-b5e4-4c5c-8706-d83184ff6d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.554649 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f765f-b5e4-4c5c-8706-d83184ff6d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:56.556248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f765f-b5e4-4c5c-8706-d83184ff6d45 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:56.567523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1245/2465] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:15:56.571648 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044503b-6f43-4e45-b2e0-0c5689e4f58b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:56.581807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044503b-6f43-4e45-b2e0-0c5689e4f58b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:56.586454 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48bdc7-dfe1-4d6a-bd2b-85d8ea87f1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:56.587141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48bdc7-dfe1-4d6a-bd2b-85d8ea87f1ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.587424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48bdc7-dfe1-4d6a-bd2b-85d8ea87f1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.587780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48bdc7-dfe1-4d6a-bd2b-85d8ea87f1ea admin admin] PATH_INFO: `/v3/projects/e37726f7fa5d46c4844e8b2b6ac846c6/users/622ecdd794af4d20816fcfde1a5bb6d5/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.603399 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48bdc7-dfe1-4d6a-bd2b-85d8ea87f1ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e37726f7fa5d46c4844e8b2b6ac846c6, user_id=622ecdd794af4d20816fcfde1a5bb6d5, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:56.606024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48bdc7-dfe1-4d6a-bd2b-85d8ea87f1ea admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:56.619457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1221/2466] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:56 2022] PUT /identity/v3/projects/e37726f7fa5d46c4844e8b2b6ac846c6/users/622ecdd794af4d20816fcfde1a5bb6d5/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:56.631147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83e55b-3f2c-45df-8353-70ccb6a7141c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.631547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83e55b-3f2c-45df-8353-70ccb6a7141c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.631815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83e55b-3f2c-45df-8353-70ccb6a7141c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.659919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8c83e55b-3f2c-45df-8353-70ccb6a7141c None None] MFA Rules not processed for user `622ecdd794af4d20816fcfde1a5bb6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:56.661147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8c83e55b-3f2c-45df-8353-70ccb6a7141c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:56.726316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1246/2467] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:56.737647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48bdc7-dfe1-4d6a-bd2b-85d8ea87f1ea admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:56.739432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1c48bdc7-dfe1-4d6a-bd2b-85d8ea87f1ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:56.761593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:56.805044 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48bdc7-dfe1-4d6a-bd2b-85d8ea87f1ea admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:56.809223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e812f5-1468-469a-bb45-7dd5a596e953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:56.809879 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e812f5-1468-469a-bb45-7dd5a596e953 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.810251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e812f5-1468-469a-bb45-7dd5a596e953 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.810583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e812f5-1468-469a-bb45-7dd5a596e953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.820312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e812f5-1468-469a-bb45-7dd5a596e953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:56.821811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e812f5-1468-469a-bb45-7dd5a596e953 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:56.832545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1222/2468] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:56.860341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83e55b-3f2c-45df-8353-70ccb6a7141c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:56.910061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:56.926131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83e55b-3f2c-45df-8353-70ccb6a7141c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:56.930770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:56.931441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:56.931726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:56.932023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] PATH_INFO: `/v3/users/622ecdd794af4d20816fcfde1a5bb6d5` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:56.933182 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] RBAC: Authorizing `identity:delete_user(user_id=622ecdd794af4d20816fcfde1a5bb6d5)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:56.935147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:56.983417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '622ecdd794af4d20816fcfde1a5bb6d5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:56.995318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '622ecdd794af4d20816fcfde1a5bb6d5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:57.002739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '622ecdd794af4d20816fcfde1a5bb6d5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:57.010885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '622ecdd794af4d20816fcfde1a5bb6d5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:57.011550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '622ecdd794af4d20816fcfde1a5bb6d5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:57.019089 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1247/2469] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:15:56 2022] DELETE /identity/v3/users/622ecdd794af4d20816fcfde1a5bb6d5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:57.023600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b37909-7469-4eef-b387-8850e070a2ea None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:57.023884 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b37909-7469-4eef-b387-8850e070a2ea None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:57.024160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b37909-7469-4eef-b387-8850e070a2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:57.042553 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-98b37909-7469-4eef-b387-8850e070a2ea None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:57.043815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-98b37909-7469-4eef-b387-8850e070a2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:57.102435 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1223/2470] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:57.112399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:57.113840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:57.172899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f66c3-f96f-424d-ac7d-7aed103bcbff admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:57.176972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9cb7e-e662-4b85-ad3d-d3b0130db04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:57.177571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9cb7e-e662-4b85-ad3d-d3b0130db04d service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:57.177849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9cb7e-e662-4b85-ad3d-d3b0130db04d service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:57.178119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9cb7e-e662-4b85-ad3d-d3b0130db04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:57.187532 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9cb7e-e662-4b85-ad3d-d3b0130db04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:57.189263 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9cb7e-e662-4b85-ad3d-d3b0130db04d service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:57.199503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1248/2471] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:57.487525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b37909-7469-4eef-b387-8850e070a2ea None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:57.489195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-98b37909-7469-4eef-b387-8850e070a2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:57.505094 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b37909-7469-4eef-b387-8850e070a2ea None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:57.509705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:57.510368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:57.510641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:57.510966 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] PATH_INFO: `/v3/projects/e37726f7fa5d46c4844e8b2b6ac846c6` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:57.512081 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e37726f7fa5d46c4844e8b2b6ac846c6)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:57.514004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:57.565189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e37726f7fa5d46c4844e8b2b6ac846c6', 'request_id': 'req-631556f7-491e-4676-b7e6-82d0185b438b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:57.570694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e37726f7fa5d46c4844e8b2b6ac846c6', 'request_id': 'req-631556f7-491e-4676-b7e6-82d0185b438b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:57.576937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e37726f7fa5d46c4844e8b2b6ac846c6', 'request_id': 'req-631556f7-491e-4676-b7e6-82d0185b438b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:57.591361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] The token cache is being invalidate because project e37726f7fa5d46c4844e8b2b6ac846c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:15:57.591672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:57.592383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-631556f7-491e-4676-b7e6-82d0185b438b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:15:57.593990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1224/2472] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:15:57 2022] DELETE /identity/v3/projects/e37726f7fa5d46c4844e8b2b6ac846c6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:57.598838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7a638-e158-44d0-addd-07d24c566b9f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:57.599335 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7a638-e158-44d0-addd-07d24c566b9f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:57.599448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7a638-e158-44d0-addd-07d24c566b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:57.619177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8ce7a638-e158-44d0-addd-07d24c566b9f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:57.620382 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8ce7a638-e158-44d0-addd-07d24c566b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:57.684514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1249/2473] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:57.692293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292cf2-5686-455c-9f83-335db41af395 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:57.692584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292cf2-5686-455c-9f83-335db41af395 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:57.692891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292cf2-5686-455c-9f83-335db41af395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:57.716153 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-fc292cf2-5686-455c-9f83-335db41af395 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:57.717639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fc292cf2-5686-455c-9f83-335db41af395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:57.726209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1225/2474] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:57.730420 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7a638-e158-44d0-addd-07d24c566b9f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:57.740921 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7a638-e158-44d0-addd-07d24c566b9f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:57.745531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce76e3-c85e-4b18-b331-86e9c6e3e670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:57.746121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce76e3-c85e-4b18-b331-86e9c6e3e670 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:57.746400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce76e3-c85e-4b18-b331-86e9c6e3e670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:57.746692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce76e3-c85e-4b18-b331-86e9c6e3e670 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:57.747365 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce76e3-c85e-4b18-b331-86e9c6e3e670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:15:57.747708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce76e3-c85e-4b18-b331-86e9c6e3e670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:57.749288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce76e3-c85e-4b18-b331-86e9c6e3e670 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:57.752248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:57.763515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1250/2475] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:15:57.768262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc292cf2-5686-455c-9f83-335db41af395 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:57.777826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc292cf2-5686-455c-9f83-335db41af395 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:57.783067 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467440f-090a-40b5-ac5a-c387a22f318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:57.783666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467440f-090a-40b5-ac5a-c387a22f318d admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:57.783947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467440f-090a-40b5-ac5a-c387a22f318d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:57.784220 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467440f-090a-40b5-ac5a-c387a22f318d admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:57.784861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467440f-090a-40b5-ac5a-c387a22f318d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:57.786768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467440f-090a-40b5-ac5a-c387a22f318d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:57.814210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1226/2476] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:15:57 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:57.818457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce76e3-c85e-4b18-b331-86e9c6e3e670 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:57.828652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce76e3-c85e-4b18-b331-86e9c6e3e670 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:57.885461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98acff-4ff8-4199-8de7-9cb0be7fa2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:57.886077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98acff-4ff8-4199-8de7-9cb0be7fa2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:57.886357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98acff-4ff8-4199-8de7-9cb0be7fa2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:57.886630 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98acff-4ff8-4199-8de7-9cb0be7fa2c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:57.887300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98acff-4ff8-4199-8de7-9cb0be7fa2c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:57.889212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98acff-4ff8-4199-8de7-9cb0be7fa2c7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:57.909906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1251/2477] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:15:57 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:15:57.914331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467440f-090a-40b5-ac5a-c387a22f318d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:57.926364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467440f-090a-40b5-ac5a-c387a22f318d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:57.930895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46005c3e-734f-42de-bec7-80b03515b6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:57.931525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46005c3e-734f-42de-bec7-80b03515b6ff admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:57.931801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46005c3e-734f-42de-bec7-80b03515b6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:57.932066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46005c3e-734f-42de-bec7-80b03515b6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:57.932699 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46005c3e-734f-42de-bec7-80b03515b6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:57.934260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46005c3e-734f-42de-bec7-80b03515b6ff admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:57.944338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1227/2478] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:15:57.949751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98acff-4ff8-4199-8de7-9cb0be7fa2c7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:57.959840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98acff-4ff8-4199-8de7-9cb0be7fa2c7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:57.964512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6dd095-21b8-44b8-b49a-c64d84321e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:57.965116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dd095-21b8-44b8-b49a-c64d84321e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:57.965394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dd095-21b8-44b8-b49a-c64d84321e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:57.965671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dd095-21b8-44b8-b49a-c64d84321e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:57.966316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6dd095-21b8-44b8-b49a-c64d84321e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:57.968449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6dd095-21b8-44b8-b49a-c64d84321e3d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:57.979091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1252/2479] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:15:57.983358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46005c3e-734f-42de-bec7-80b03515b6ff admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:57.993794 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46005c3e-734f-42de-bec7-80b03515b6ff admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:57.998348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528d912-50bc-4f73-8792-775704c0206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:57.999040 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528d912-50bc-4f73-8792-775704c0206e admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:57.999332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528d912-50bc-4f73-8792-775704c0206e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:57.999607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528d912-50bc-4f73-8792-775704c0206e admin admin] PATH_INFO: `/v3/projects/1443ca94374d4d3e89165346152d76e1/users/015a74f1c0194f918e65723d4ec4d904/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:58.001853 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:15:58.017324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528d912-50bc-4f73-8792-775704c0206e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1443ca94374d4d3e89165346152d76e1, user_id=015a74f1c0194f918e65723d4ec4d904, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:58.020146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528d912-50bc-4f73-8792-775704c0206e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:58.034202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1228/2480] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:15:57 2022] PUT /identity/v3/projects/1443ca94374d4d3e89165346152d76e1/users/015a74f1c0194f918e65723d4ec4d904/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:15:58.045167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d17dd-b9ab-4612-91e1-6d9449c19fe3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:58.045461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d17dd-b9ab-4612-91e1-6d9449c19fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:58.045738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d17dd-b9ab-4612-91e1-6d9449c19fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:58.073970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5b8d17dd-b9ab-4612-91e1-6d9449c19fe3 None None] MFA Rules not processed for user `015a74f1c0194f918e65723d4ec4d904`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:58.075217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5b8d17dd-b9ab-4612-91e1-6d9449c19fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:58.140744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1253/2481] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:15:58.152070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528d912-50bc-4f73-8792-775704c0206e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:15:58.153842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2528d912-50bc-4f73-8792-775704c0206e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:58.210791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528d912-50bc-4f73-8792-775704c0206e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:15:58.214678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25a6a1-416d-4f13-ac81-72d6d289731a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:15:58.215324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25a6a1-416d-4f13-ac81-72d6d289731a service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:58.215604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25a6a1-416d-4f13-ac81-72d6d289731a service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:58.215870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25a6a1-416d-4f13-ac81-72d6d289731a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:58.225120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25a6a1-416d-4f13-ac81-72d6d289731a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:15:58.226546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25a6a1-416d-4f13-ac81-72d6d289731a service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:15:58.236545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1229/2482] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:15:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:15:59.600376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28986d85-1dee-4429-9002-40354afa7447 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:15:59.600906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28986d85-1dee-4429-9002-40354afa7447 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:15:59.600906 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28986d85-1dee-4429-9002-40354afa7447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:15:59.619702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-28986d85-1dee-4429-9002-40354afa7447 None None] MFA Rules not processed for user `015a74f1c0194f918e65723d4ec4d904`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:15:59.620904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-28986d85-1dee-4429-9002-40354afa7447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:15:59.628625 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1254/2483] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:15:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:16:08.231817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:09.614277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:16.657804 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d41248-e6cc-4338-9c8a-aec03893b81a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:16.658313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d41248-e6cc-4338-9c8a-aec03893b81a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:16.658453 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d41248-e6cc-4338-9c8a-aec03893b81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:16.677558 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-84d41248-e6cc-4338-9c8a-aec03893b81a None None] MFA Rules not processed for user `5feea3b3b7e4401ea7db242cb60fdfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:16:16.678758 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-84d41248-e6cc-4338-9c8a-aec03893b81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:16.735016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1230/2484] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:16:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:16:18.649423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675baf45-b181-452a-bb11-63f2d63cf1c3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:18.649912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675baf45-b181-452a-bb11-63f2d63cf1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:18.649996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675baf45-b181-452a-bb11-63f2d63cf1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:18.670647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-675baf45-b181-452a-bb11-63f2d63cf1c3 None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:16:18.672011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-675baf45-b181-452a-bb11-63f2d63cf1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:18.733652 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1255/2485] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:16:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:16:18.781096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d41248-e6cc-4338-9c8a-aec03893b81a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:18.782551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-84d41248-e6cc-4338-9c8a-aec03893b81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:18.853019 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d41248-e6cc-4338-9c8a-aec03893b81a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:18.858521 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37e35e-3e1d-47dc-b0cd-ec5a22058b11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:18.859156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37e35e-3e1d-47dc-b0cd-ec5a22058b11 service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:18.859433 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37e35e-3e1d-47dc-b0cd-ec5a22058b11 service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:18.859707 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37e35e-3e1d-47dc-b0cd-ec5a22058b11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:18.868290 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37e35e-3e1d-47dc-b0cd-ec5a22058b11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:18.869886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37e35e-3e1d-47dc-b0cd-ec5a22058b11 service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:18.881689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1231/2486] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:16:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:16:19.416758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c315e-9475-4b47-a196-ffbb140506f8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:19.417048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c315e-9475-4b47-a196-ffbb140506f8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:19.417327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c315e-9475-4b47-a196-ffbb140506f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:19.438289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-948c315e-9475-4b47-a196-ffbb140506f8 None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:16:19.440875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-948c315e-9475-4b47-a196-ffbb140506f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:19.447638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1256/2487] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 22 17:16:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:16:19.460824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37e35e-3e1d-47dc-b0cd-ec5a22058b11 service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:19.471478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37e35e-3e1d-47dc-b0cd-ec5a22058b11 service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:19.478609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fe6d4-ff00-4367-b4f5-1fa87881ec59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:19.479245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fe6d4-ff00-4367-b4f5-1fa87881ec59 service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:19.479527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fe6d4-ff00-4367-b4f5-1fa87881ec59 service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:19.479800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fe6d4-ff00-4367-b4f5-1fa87881ec59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:19.488289 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fe6d4-ff00-4367-b4f5-1fa87881ec59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:19.489867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fe6d4-ff00-4367-b4f5-1fa87881ec59 service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:19.498835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1232/2488] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:16:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:16:19.737924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd04117-201f-41d6-8596-c6d1d4104a95 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:19.738454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd04117-201f-41d6-8596-c6d1d4104a95 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:19.738454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd04117-201f-41d6-8596-c6d1d4104a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:19.762270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6bd04117-201f-41d6-8596-c6d1d4104a95 None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:16:19.766167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6bd04117-201f-41d6-8596-c6d1d4104a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:19.774262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1257/2489] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:16:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:16:19.787092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fe6d4-ff00-4367-b4f5-1fa87881ec59 service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:19.797848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fe6d4-ff00-4367-b4f5-1fa87881ec59 service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:19.802941 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2493ad-d873-4c51-8619-1a373d040bcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:19.803574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2493ad-d873-4c51-8619-1a373d040bcb service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:19.803846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2493ad-d873-4c51-8619-1a373d040bcb service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:19.804115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2493ad-d873-4c51-8619-1a373d040bcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:19.812768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2493ad-d873-4c51-8619-1a373d040bcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:19.814734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2493ad-d873-4c51-8619-1a373d040bcb service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:19.824880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1233/2490] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:16:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:16:20.213545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2575df0-988f-42e0-87cf-aa874c3a64df None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:20.213833 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2575df0-988f-42e0-87cf-aa874c3a64df None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:20.214106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2575df0-988f-42e0-87cf-aa874c3a64df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:20.233680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b2575df0-988f-42e0-87cf-aa874c3a64df None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:16:20.235031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b2575df0-988f-42e0-87cf-aa874c3a64df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:20.243176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1258/2491] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 22 17:16:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:16:20.256694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2493ad-d873-4c51-8619-1a373d040bcb service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:20.268544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2493ad-d873-4c51-8619-1a373d040bcb service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:20.273623 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dac47-1161-44cd-be23-ee729c533ad8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:20.274228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dac47-1161-44cd-be23-ee729c533ad8 service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:20.274538 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dac47-1161-44cd-be23-ee729c533ad8 service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:20.274812 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dac47-1161-44cd-be23-ee729c533ad8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:20.283231 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806dac47-1161-44cd-be23-ee729c533ad8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:20.284804 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806dac47-1161-44cd-be23-ee729c533ad8 service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:20.293179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1234/2492] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:16:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:16:22.473249 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2575df0-988f-42e0-87cf-aa874c3a64df None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:22.539252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2575df0-988f-42e0-87cf-aa874c3a64df None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:22.544437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:22.545079 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:22.545386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:22.545657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] PATH_INFO: `/v3/users/015a74f1c0194f918e65723d4ec4d904` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:22.546909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=015a74f1c0194f918e65723d4ec4d904)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:22.549999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:22.606725 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '015a74f1c0194f918e65723d4ec4d904'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:22.620174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '015a74f1c0194f918e65723d4ec4d904'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:22.627146 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '015a74f1c0194f918e65723d4ec4d904'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:22.634896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '015a74f1c0194f918e65723d4ec4d904'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:22.635572 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '015a74f1c0194f918e65723d4ec4d904'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:22.637008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:22.643159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1259/2493] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:16:22 2022] DELETE /identity/v3/users/015a74f1c0194f918e65723d4ec4d904 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:16:22.649219 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d219f3-11ff-4cac-84a3-7e546b7068b8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:22.650196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d219f3-11ff-4cac-84a3-7e546b7068b8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:22.650672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d219f3-11ff-4cac-84a3-7e546b7068b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:22.671830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-14d219f3-11ff-4cac-84a3-7e546b7068b8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:16:22.673441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-14d219f3-11ff-4cac-84a3-7e546b7068b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:22.719491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:22.730706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1235/2494] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:16:22.741064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:22.742772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:22.801252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4123d-bb9c-466b-8e33-8a129d0c9062 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:22.805535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059c8d4-1307-4e82-a596-37850177a927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:22.806148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059c8d4-1307-4e82-a596-37850177a927 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:22.806434 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059c8d4-1307-4e82-a596-37850177a927 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:22.806709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059c8d4-1307-4e82-a596-37850177a927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:22.815946 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059c8d4-1307-4e82-a596-37850177a927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:22.817368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059c8d4-1307-4e82-a596-37850177a927 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:22.827388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1260/2495] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:16:22.997820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d219f3-11ff-4cac-84a3-7e546b7068b8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:22.999262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-14d219f3-11ff-4cac-84a3-7e546b7068b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:23.011066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d219f3-11ff-4cac-84a3-7e546b7068b8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:23.016262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:23.016890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:23.017162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:23.017549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] PATH_INFO: `/v3/projects/1443ca94374d4d3e89165346152d76e1` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:23.018848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1443ca94374d4d3e89165346152d76e1)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:23.020769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:23.067729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1443ca94374d4d3e89165346152d76e1', 'request_id': 'req-97c4359b-f6d7-4b2e-9bc7-683b22862937'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:23.073257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1443ca94374d4d3e89165346152d76e1', 'request_id': 'req-97c4359b-f6d7-4b2e-9bc7-683b22862937'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:23.079984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1443ca94374d4d3e89165346152d76e1', 'request_id': 'req-97c4359b-f6d7-4b2e-9bc7-683b22862937'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:23.092511 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] The token cache is being invalidate because project 1443ca94374d4d3e89165346152d76e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:16:23.092795 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:23.093424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:23.094869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1236/2496] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:16:22 2022] DELETE /identity/v3/projects/1443ca94374d4d3e89165346152d76e1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:16:32.822595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=27 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:33.088840 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:39.432058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d749530-b923-4006-880f-fb66ea7ed8fe None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:39.432716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d749530-b923-4006-880f-fb66ea7ed8fe None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:39.432716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d749530-b923-4006-880f-fb66ea7ed8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:39.453324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6d749530-b923-4006-880f-fb66ea7ed8fe None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:16:39.454546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6d749530-b923-4006-880f-fb66ea7ed8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:39.520071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1261/2497] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 22 17:16:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:16:39.533527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:39.535057 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:39.604593 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4359b-f6d7-4b2e-9bc7-683b22862937 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:39.609793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc072c8-3d02-4cc8-a1a3-7d048c821949 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:39.610415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc072c8-3d02-4cc8-a1a3-7d048c821949 service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:39.610698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc072c8-3d02-4cc8-a1a3-7d048c821949 service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:39.610997 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc072c8-3d02-4cc8-a1a3-7d048c821949 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:39.619665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc072c8-3d02-4cc8-a1a3-7d048c821949 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:39.621305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc072c8-3d02-4cc8-a1a3-7d048c821949 service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:39.629937 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1237/2498] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:16:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:16:41.318647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d749530-b923-4006-880f-fb66ea7ed8fe None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:41.320713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6d749530-b923-4006-880f-fb66ea7ed8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:41.388776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d749530-b923-4006-880f-fb66ea7ed8fe None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:41.392996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a38e9-63be-44ca-975b-b15d33cdd291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:41.393611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a38e9-63be-44ca-975b-b15d33cdd291 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:41.393889 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a38e9-63be-44ca-975b-b15d33cdd291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:41.394163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a38e9-63be-44ca-975b-b15d33cdd291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:41.395972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ab1a38e9-63be-44ca-975b-b15d33cdd291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:41.471776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a38e9-63be-44ca-975b-b15d33cdd291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:41.473224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a38e9-63be-44ca-975b-b15d33cdd291 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:41.484886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1262/2499] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:16:43.407745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc072c8-3d02-4cc8-a1a3-7d048c821949 service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:43.409763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6fc072c8-3d02-4cc8-a1a3-7d048c821949 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:43.543945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc072c8-3d02-4cc8-a1a3-7d048c821949 service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:43.550144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13aa7fc-c9e9-4645-8248-19f86aec37ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:43.550773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13aa7fc-c9e9-4645-8248-19f86aec37ce service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:43.551294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13aa7fc-c9e9-4645-8248-19f86aec37ce service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:43.551723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13aa7fc-c9e9-4645-8248-19f86aec37ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:43.553684 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a13aa7fc-c9e9-4645-8248-19f86aec37ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:43.568051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13aa7fc-c9e9-4645-8248-19f86aec37ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:43.570328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13aa7fc-c9e9-4645-8248-19f86aec37ce service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:43.584043 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1238/2500] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:16:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:16:46.679406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651b874-f067-42fa-a7ba-d9d99428aafd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:46.679960 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651b874-f067-42fa-a7ba-d9d99428aafd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:46.680171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651b874-f067-42fa-a7ba-d9d99428aafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:46.706239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0651b874-f067-42fa-a7ba-d9d99428aafd None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:16:46.707845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0651b874-f067-42fa-a7ba-d9d99428aafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:46.715728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1263/2501] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:16:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:16:46.737597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13aa7fc-c9e9-4645-8248-19f86aec37ce service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:46.750535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13aa7fc-c9e9-4645-8248-19f86aec37ce service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:46.759958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee9b4f-2b52-4989-a010-b9393e7e29a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:46.761037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee9b4f-2b52-4989-a010-b9393e7e29a1 service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:46.761381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee9b4f-2b52-4989-a010-b9393e7e29a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:46.761785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee9b4f-2b52-4989-a010-b9393e7e29a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:46.773268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee9b4f-2b52-4989-a010-b9393e7e29a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:46.776675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee9b4f-2b52-4989-a010-b9393e7e29a1 service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:46.793990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1239/2502] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:16:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:16:48.745006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651b874-f067-42fa-a7ba-d9d99428aafd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:48.747242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0651b874-f067-42fa-a7ba-d9d99428aafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:48.774167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651b874-f067-42fa-a7ba-d9d99428aafd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:48.787119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:48.788002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:48.788452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:48.789486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] PATH_INFO: `/v3/users/cf878d3df653463caa7767de835dce83` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:48.791234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf878d3df653463caa7767de835dce83)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:48.793395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:48.894701 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:48.900081 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf878d3df653463caa7767de835dce83'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:48.937304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf878d3df653463caa7767de835dce83'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:48.946796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf878d3df653463caa7767de835dce83'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:48.957829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf878d3df653463caa7767de835dce83'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:48.959610 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf878d3df653463caa7767de835dce83'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:48.987462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1264/2503] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:16:48 2022] DELETE /identity/v3/users/cf878d3df653463caa7767de835dce83 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:16:49.474802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee9b4f-2b52-4989-a010-b9393e7e29a1 service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:49.486733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee9b4f-2b52-4989-a010-b9393e7e29a1 service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:49.595617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:49.596405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:49.599426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:49.600450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] PATH_INFO: `/v3/projects/0749b2616b854791aec389d669ef94c8` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:49.601806 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0749b2616b854791aec389d669ef94c8)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:49.604598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:49.628859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:49.748211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0749b2616b854791aec389d669ef94c8', 'request_id': 'req-65145db4-5adb-485a-92d6-74436ccd7ae0'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:49.774341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0749b2616b854791aec389d669ef94c8', 'request_id': 'req-65145db4-5adb-485a-92d6-74436ccd7ae0'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:49.783215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0749b2616b854791aec389d669ef94c8', 'request_id': 'req-65145db4-5adb-485a-92d6-74436ccd7ae0'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:49.804829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] The token cache is being invalidate because project 0749b2616b854791aec389d669ef94c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:16:49.805320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:49.806189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:49.809000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1240/2504] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:16:49 2022] DELETE /identity/v3/projects/0749b2616b854791aec389d669ef94c8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:16:49.813214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:49.814764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:49.832879 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d66b8-4138-4afb-ade8-9e1c5b17ccb1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:49.938939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:49.939764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:49.940151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:49.940533 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] PATH_INFO: `/v3/users/b34f97f989d44b879db0854801e272c9` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:49.944388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b34f97f989d44b879db0854801e272c9)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:49.946471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:50.023838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b34f97f989d44b879db0854801e272c9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:50.052636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b34f97f989d44b879db0854801e272c9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:50.077883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b34f97f989d44b879db0854801e272c9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:50.090280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b34f97f989d44b879db0854801e272c9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:50.091386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b34f97f989d44b879db0854801e272c9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:50.099147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1265/2505] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:16:49 2022] DELETE /identity/v3/users/b34f97f989d44b879db0854801e272c9 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:16:50.437316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:50.439135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:50.478332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145db4-5adb-485a-92d6-74436ccd7ae0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:50.610911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:50.611665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:50.612102 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:50.612441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] PATH_INFO: `/v3/projects/717ff3c230e549f684162b5fed430228` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:50.613751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=717ff3c230e549f684162b5fed430228)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:50.616865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:50.713825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '717ff3c230e549f684162b5fed430228', 'request_id': 'req-52797d39-d9c8-4e22-81d8-477c91e0e3cf'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:50.739650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '717ff3c230e549f684162b5fed430228', 'request_id': 'req-52797d39-d9c8-4e22-81d8-477c91e0e3cf'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:50.751510 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '717ff3c230e549f684162b5fed430228', 'request_id': 'req-52797d39-d9c8-4e22-81d8-477c91e0e3cf'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:50.784464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] The token cache is being invalidate because project 717ff3c230e549f684162b5fed430228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:16:50.784991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:50.785834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:50.788419 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1241/2506] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:16:50 2022] DELETE /identity/v3/projects/717ff3c230e549f684162b5fed430228 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:16:51.449791 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:51.452365 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:51.467466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00af60-1dfa-43df-8681-8f58c61133e0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:51.510426 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:51.534131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:51.534930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:51.535309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:51.535656 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] PATH_INFO: `/v3/users/0365ab1bcfd54946b0b92f8fe9697128` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:51.536805 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0365ab1bcfd54946b0b92f8fe9697128)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:51.538868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:51.599535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0365ab1bcfd54946b0b92f8fe9697128'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:51.613953 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0365ab1bcfd54946b0b92f8fe9697128'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:51.624006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0365ab1bcfd54946b0b92f8fe9697128'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:51.633118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0365ab1bcfd54946b0b92f8fe9697128'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:51.633756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0365ab1bcfd54946b0b92f8fe9697128'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:51.641826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1266/2507] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:16:51 2022] DELETE /identity/v3/users/0365ab1bcfd54946b0b92f8fe9697128 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:16:51.854017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:51.856914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:51.894786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52797d39-d9c8-4e22-81d8-477c91e0e3cf admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:51.933406 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:51.976586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:51.977314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:51.977661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:51.978025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] PATH_INFO: `/v3/projects/1ed7377d32844bbf95f995831bc21d6b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:51.979503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ed7377d32844bbf95f995831bc21d6b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:51.981776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:52.042690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ed7377d32844bbf95f995831bc21d6b', 'request_id': 'req-bfc2d46b-6563-4f46-8f40-1d0c133aa049'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.050222 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ed7377d32844bbf95f995831bc21d6b', 'request_id': 'req-bfc2d46b-6563-4f46-8f40-1d0c133aa049'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.057906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ed7377d32844bbf95f995831bc21d6b', 'request_id': 'req-bfc2d46b-6563-4f46-8f40-1d0c133aa049'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.077933 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] The token cache is being invalidate because project 1ed7377d32844bbf95f995831bc21d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:16:52.078224 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.078865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.080449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1242/2508] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:16:51 2022] DELETE /identity/v3/projects/1ed7377d32844bbf95f995831bc21d6b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:16:52.084540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:52.086270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:52.100962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd5ad0-18e9-4b18-baf7-683f7fb94b60 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:52.172172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b92ba-8389-4c3c-8d1a-ebe361e7fade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:52.173074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b92ba-8389-4c3c-8d1a-ebe361e7fade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:52.173361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b92ba-8389-4c3c-8d1a-ebe361e7fade admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:52.173634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b92ba-8389-4c3c-8d1a-ebe361e7fade admin admin] PATH_INFO: `/v3/users/636a32ea73fe4987947708bc1862aba2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:52.174857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b92ba-8389-4c3c-8d1a-ebe361e7fade admin admin] RBAC: Authorizing `identity:delete_user(user_id=636a32ea73fe4987947708bc1862aba2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:52.178437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b92ba-8389-4c3c-8d1a-ebe361e7fade admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:52.236848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ee5b92ba-8389-4c3c-8d1a-ebe361e7fade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '636a32ea73fe4987947708bc1862aba2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.251002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:52.261895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ee5b92ba-8389-4c3c-8d1a-ebe361e7fade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '636a32ea73fe4987947708bc1862aba2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.271417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ee5b92ba-8389-4c3c-8d1a-ebe361e7fade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '636a32ea73fe4987947708bc1862aba2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.281735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ee5b92ba-8389-4c3c-8d1a-ebe361e7fade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '636a32ea73fe4987947708bc1862aba2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.282410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ee5b92ba-8389-4c3c-8d1a-ebe361e7fade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '636a32ea73fe4987947708bc1862aba2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.288938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1267/2509] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:16:52 2022] DELETE /identity/v3/users/636a32ea73fe4987947708bc1862aba2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:16:52.540909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:16:52.542588 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:16:52.565895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2d46b-6563-4f46-8f40-1d0c133aa049 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:16:52.638377 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:16:52.639730 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:16:52.640074 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:16:52.640560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] PATH_INFO: `/v3/projects/5bf728292e2a4eb2bd1097b01fa6533b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:16:52.642639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf728292e2a4eb2bd1097b01fa6533b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:16:52.644578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:16:52.711248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf728292e2a4eb2bd1097b01fa6533b', 'request_id': 'req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.718315 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf728292e2a4eb2bd1097b01fa6533b', 'request_id': 'req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.720218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:16:52.728219 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf728292e2a4eb2bd1097b01fa6533b', 'request_id': 'req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.744341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] The token cache is being invalidate because project 5bf728292e2a4eb2bd1097b01fa6533b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:16:52.745075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.745725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6308cfd7-428a-42b3-8db3-6eb8e325e9d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:16:52.747684 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1243/2510] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:16:52 2022] DELETE /identity/v3/projects/5bf728292e2a4eb2bd1097b01fa6533b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:17:02.285350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:17:02.740749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=4,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:17:03.052427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c6b6c-0c7c-467f-ae8d-c261303c2dd9 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:03.052631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c6b6c-0c7c-467f-ae8d-c261303c2dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:03.052899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c6b6c-0c7c-467f-ae8d-c261303c2dd9 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:03.054288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1268/2511] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 22 17:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 17:17:03.059319 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49cb1c-0b42-45a9-b192-060f84521fea None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:03.059690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49cb1c-0b42-45a9-b192-060f84521fea None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:03.060025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49cb1c-0b42-45a9-b192-060f84521fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:03.093845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1b49cb1c-0b42-45a9-b192-060f84521fea None None] MFA Rules not processed for user `f6c1ce47d8704111988a78e625fd968c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:17:03.095102 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1b49cb1c-0b42-45a9-b192-060f84521fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:03.174291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1244/2512] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 22 17:17:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:17:03.186565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c6b6c-0c7c-467f-ae8d-c261303c2dd9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:03.188396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-986c6b6c-0c7c-467f-ae8d-c261303c2dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:03.254782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c6b6c-0c7c-467f-ae8d-c261303c2dd9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:03.258437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcc9aa-8f79-4855-9f0f-0fd326b0e216 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6103e9a612c84a9ea7d21cc5128fa40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:03.259088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcc9aa-8f79-4855-9f0f-0fd326b0e216 service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:03.259371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcc9aa-8f79-4855-9f0f-0fd326b0e216 service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:03.259646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcc9aa-8f79-4855-9f0f-0fd326b0e216 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:03.271818 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcc9aa-8f79-4855-9f0f-0fd326b0e216 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:03.273314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcc9aa-8f79-4855-9f0f-0fd326b0e216 service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:03.286334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1269/2513] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:17:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:17:05.358875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261dba-314b-46c3-98d8-d9038b9a903d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:05.359467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261dba-314b-46c3-98d8-d9038b9a903d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:05.359577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261dba-314b-46c3-98d8-d9038b9a903d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:05.380505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f2261dba-314b-46c3-98d8-d9038b9a903d None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:17:05.381881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f2261dba-314b-46c3-98d8-d9038b9a903d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:05.438862 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1245/2514] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 22 17:17:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:17:05.451107 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcc9aa-8f79-4855-9f0f-0fd326b0e216 service glance] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:05.452526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-40fcc9aa-8f79-4855-9f0f-0fd326b0e216 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:05.531636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcc9aa-8f79-4855-9f0f-0fd326b0e216 service glance] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:05.537289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756399ec-c353-461d-8388-24ac35fec5f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:05.537905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756399ec-c353-461d-8388-24ac35fec5f5 service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:05.538180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756399ec-c353-461d-8388-24ac35fec5f5 service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:05.538453 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756399ec-c353-461d-8388-24ac35fec5f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:05.548224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756399ec-c353-461d-8388-24ac35fec5f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:05.549803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756399ec-c353-461d-8388-24ac35fec5f5 service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:05.558375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1270/2515] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:17:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 17:17:07.036826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2261dba-314b-46c3-98d8-d9038b9a903d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:07.038179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f2261dba-314b-46c3-98d8-d9038b9a903d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:07.098401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2261dba-314b-46c3-98d8-d9038b9a903d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:07.101972 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d7d1f-2f5f-48ff-b1cf-268bca47896d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:07.102578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d7d1f-2f5f-48ff-b1cf-268bca47896d service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:07.102849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d7d1f-2f5f-48ff-b1cf-268bca47896d service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:07.103155 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d7d1f-2f5f-48ff-b1cf-268bca47896d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:07.104900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5f4d7d1f-2f5f-48ff-b1cf-268bca47896d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:07.120136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d7d1f-2f5f-48ff-b1cf-268bca47896d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:07.120458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d7d1f-2f5f-48ff-b1cf-268bca47896d service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:07.134481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1246/2516] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:17:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:17:08.395047 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756399ec-c353-461d-8388-24ac35fec5f5 service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:08.409384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756399ec-c353-461d-8388-24ac35fec5f5 service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:08.413417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965832d-a66d-467b-9031-2f3400a5eb64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:08.414058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965832d-a66d-467b-9031-2f3400a5eb64 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:08.414314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965832d-a66d-467b-9031-2f3400a5eb64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:08.414599 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965832d-a66d-467b-9031-2f3400a5eb64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:08.416453 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6965832d-a66d-467b-9031-2f3400a5eb64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:08.431979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965832d-a66d-467b-9031-2f3400a5eb64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:08.433468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965832d-a66d-467b-9031-2f3400a5eb64 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:08.445156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1271/2517] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:17:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:17:17.129203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:17:18.440008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=64 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:17:27.832215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8264b61-d8be-483b-b6aa-bca0c059af71 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:27.832725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8264b61-d8be-483b-b6aa-bca0c059af71 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:27.832829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8264b61-d8be-483b-b6aa-bca0c059af71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:27.856946 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b8264b61-d8be-483b-b6aa-bca0c059af71 None None] MFA Rules not processed for user `abbbe2b3f23b4c1caf8e20825cc2cfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:17:27.858941 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b8264b61-d8be-483b-b6aa-bca0c059af71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:27.867194 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1247/2518] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 22 17:17:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:17:27.890717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965832d-a66d-467b-9031-2f3400a5eb64 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:27.903447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965832d-a66d-467b-9031-2f3400a5eb64 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:27.909962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ae848-968e-4279-8765-1eede1748223 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a54beb68ba4610976524684bc26d5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:27.910598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ae848-968e-4279-8765-1eede1748223 service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:27.911051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ae848-968e-4279-8765-1eede1748223 service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:27.911620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ae848-968e-4279-8765-1eede1748223 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:27.921491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ae848-968e-4279-8765-1eede1748223 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:27.923121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ae848-968e-4279-8765-1eede1748223 service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:27.932344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1272/2519] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 22 17:17:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 17:17:37.850964 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:17:37.928670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=6 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:17:45.880828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8264b61-d8be-483b-b6aa-bca0c059af71 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:45.890610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8264b61-d8be-483b-b6aa-bca0c059af71 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:45.894370 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d090539-b219-4dd5-beff-ab969a377cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:45.895048 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d090539-b219-4dd5-beff-ab969a377cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:45.895330 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d090539-b219-4dd5-beff-ab969a377cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:45.895608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d090539-b219-4dd5-beff-ab969a377cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:45.897393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9d090539-b219-4dd5-beff-ab969a377cd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:45.971439 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d090539-b219-4dd5-beff-ab969a377cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:45.972877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d090539-b219-4dd5-beff-ab969a377cd0 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:45.983104 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1248/2520] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:17:49.467514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ae848-968e-4279-8765-1eede1748223 service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:49.469058 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-941ae848-968e-4279-8765-1eede1748223 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:49.483495 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ae848-968e-4279-8765-1eede1748223 service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:49.488314 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a5a4d-b0bb-4355-8a8e-d99a18839f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:49.488938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a5a4d-b0bb-4355-8a8e-d99a18839f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:49.489212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a5a4d-b0bb-4355-8a8e-d99a18839f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:49.489480 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a5a4d-b0bb-4355-8a8e-d99a18839f38 admin admin] PATH_INFO: `/v3/users/624f8fd9a74e48afa2f8b74b090a8dd6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:49.504675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a5a4d-b0bb-4355-8a8e-d99a18839f38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=624f8fd9a74e48afa2f8b74b090a8dd6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:49.506604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a5a4d-b0bb-4355-8a8e-d99a18839f38 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:49.555596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7f1a5a4d-b0bb-4355-8a8e-d99a18839f38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624f8fd9a74e48afa2f8b74b090a8dd6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:17:49.567824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7f1a5a4d-b0bb-4355-8a8e-d99a18839f38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624f8fd9a74e48afa2f8b74b090a8dd6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:17:49.574563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7f1a5a4d-b0bb-4355-8a8e-d99a18839f38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624f8fd9a74e48afa2f8b74b090a8dd6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:17:49.582590 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7f1a5a4d-b0bb-4355-8a8e-d99a18839f38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624f8fd9a74e48afa2f8b74b090a8dd6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:17:49.583241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7f1a5a4d-b0bb-4355-8a8e-d99a18839f38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624f8fd9a74e48afa2f8b74b090a8dd6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:17:49.589162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1273/2521] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:17:49 2022] DELETE /identity/v3/users/624f8fd9a74e48afa2f8b74b090a8dd6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:17:49.768125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d090539-b219-4dd5-beff-ab969a377cd0 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:49.784086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d090539-b219-4dd5-beff-ab969a377cd0 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:49.842369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:49.842928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:49.843233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:49.843505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] PATH_INFO: `/v3/projects/2abeb0b54520461cb414ea9c0599380f` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:49.853767 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2abeb0b54520461cb414ea9c0599380f)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:49.855710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:49.901751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2abeb0b54520461cb414ea9c0599380f', 'request_id': 'req-26885d32-8c64-4924-a733-06c2c56d80b2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:17:49.907387 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2abeb0b54520461cb414ea9c0599380f', 'request_id': 'req-26885d32-8c64-4924-a733-06c2c56d80b2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:17:49.914557 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2abeb0b54520461cb414ea9c0599380f', 'request_id': 'req-26885d32-8c64-4924-a733-06c2c56d80b2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:17:49.927512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] The token cache is being invalidate because project 2abeb0b54520461cb414ea9c0599380f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:17:49.927803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:17:49.928421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26885d32-8c64-4924-a733-06c2c56d80b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:17:49.929922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1249/2522] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:17:49 2022] DELETE /identity/v3/projects/2abeb0b54520461cb414ea9c0599380f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:17:49.934844 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583e474-55f4-44f1-a7ce-8bf558b30705 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:49.935159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583e474-55f4-44f1-a7ce-8bf558b30705 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:49.935435 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583e474-55f4-44f1-a7ce-8bf558b30705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:49.955412 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5583e474-55f4-44f1-a7ce-8bf558b30705 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:17:49.956604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5583e474-55f4-44f1-a7ce-8bf558b30705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:50.015277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1274/2523] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:17:50.024075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd5b1d-7ec7-49a6-aded-8956e0d661ed None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.024367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd5b1d-7ec7-49a6-aded-8956e0d661ed None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.024652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd5b1d-7ec7-49a6-aded-8956e0d661ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.043993 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b3bd5b1d-7ec7-49a6-aded-8956e0d661ed None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:17:50.045704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b3bd5b1d-7ec7-49a6-aded-8956e0d661ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:50.053974 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1250/2524] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:17:50.058183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583e474-55f4-44f1-a7ce-8bf558b30705 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.068675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583e474-55f4-44f1-a7ce-8bf558b30705 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.073293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a3b84-27c4-4772-a72d-198b49841759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.073893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a3b84-27c4-4772-a72d-198b49841759 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.074176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a3b84-27c4-4772-a72d-198b49841759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.074446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a3b84-27c4-4772-a72d-198b49841759 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.075205 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a3b84-27c4-4772-a72d-198b49841759 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:17:50.075496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a3b84-27c4-4772-a72d-198b49841759 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.077103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a3b84-27c4-4772-a72d-198b49841759 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.088661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1275/2525] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:17:50.093467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd5b1d-7ec7-49a6-aded-8956e0d661ed None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.102534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd5b1d-7ec7-49a6-aded-8956e0d661ed None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.107212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d56268-39f3-401e-8ac4-42358ec00bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.108432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d56268-39f3-401e-8ac4-42358ec00bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.108713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d56268-39f3-401e-8ac4-42358ec00bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.108985 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d56268-39f3-401e-8ac4-42358ec00bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.109644 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d56268-39f3-401e-8ac4-42358ec00bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.111617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d56268-39f3-401e-8ac4-42358ec00bd8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.116878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:17:50.141627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1251/2526] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:17:50 2022] POST /identity/v3/projects => generated 353 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:17:50.145918 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a3b84-27c4-4772-a72d-198b49841759 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.156476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a3b84-27c4-4772-a72d-198b49841759 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.205700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:17:50.222341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5907a81-2652-4d4a-9a12-d2a78e0199a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.223017 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5907a81-2652-4d4a-9a12-d2a78e0199a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.223300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5907a81-2652-4d4a-9a12-d2a78e0199a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.223573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5907a81-2652-4d4a-9a12-d2a78e0199a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.224275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5907a81-2652-4d4a-9a12-d2a78e0199a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.226125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5907a81-2652-4d4a-9a12-d2a78e0199a9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.252330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1276/2527] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:17:50 2022] POST /identity/v3/users => generated 277 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:17:50.253544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d56268-39f3-401e-8ac4-42358ec00bd8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.264044 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d56268-39f3-401e-8ac4-42358ec00bd8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.269082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275eaa0-8f71-435c-89f1-cbdd663325aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.269697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275eaa0-8f71-435c-89f1-cbdd663325aa admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.270065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275eaa0-8f71-435c-89f1-cbdd663325aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.270513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275eaa0-8f71-435c-89f1-cbdd663325aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.271209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275eaa0-8f71-435c-89f1-cbdd663325aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.272813 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275eaa0-8f71-435c-89f1-cbdd663325aa admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.284345 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1252/2528] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:17:50.288298 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5907a81-2652-4d4a-9a12-d2a78e0199a9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.303700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5907a81-2652-4d4a-9a12-d2a78e0199a9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.308658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8fe97-a190-4017-94f7-71a90fe87c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.309269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8fe97-a190-4017-94f7-71a90fe87c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.309549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8fe97-a190-4017-94f7-71a90fe87c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.309839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8fe97-a190-4017-94f7-71a90fe87c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.310493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8fe97-a190-4017-94f7-71a90fe87c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.312103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8fe97-a190-4017-94f7-71a90fe87c61 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.323109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1277/2529] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:17:50.327567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275eaa0-8f71-435c-89f1-cbdd663325aa admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.337597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275eaa0-8f71-435c-89f1-cbdd663325aa admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.342810 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e16da-e398-4bc1-9ee2-8cd0d542be3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.343528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e16da-e398-4bc1-9ee2-8cd0d542be3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.343750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e16da-e398-4bc1-9ee2-8cd0d542be3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.344015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e16da-e398-4bc1-9ee2-8cd0d542be3a admin admin] PATH_INFO: `/v3/projects/7b3862a3adf5486e96bc9292f6296e38/users/68fe760a598b46e6a7a4a21261940e32/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.358847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e16da-e398-4bc1-9ee2-8cd0d542be3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3862a3adf5486e96bc9292f6296e38, user_id=68fe760a598b46e6a7a4a21261940e32, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.361508 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e16da-e398-4bc1-9ee2-8cd0d542be3a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.377291 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1253/2530] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:17:50 2022] PUT /identity/v3/projects/7b3862a3adf5486e96bc9292f6296e38/users/68fe760a598b46e6a7a4a21261940e32/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:17:50.387797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59df72-9e4a-4ef4-8fa2-fc147a5fff36 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.388088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59df72-9e4a-4ef4-8fa2-fc147a5fff36 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.388366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59df72-9e4a-4ef4-8fa2-fc147a5fff36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.417892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0d59df72-9e4a-4ef4-8fa2-fc147a5fff36 None None] MFA Rules not processed for user `68fe760a598b46e6a7a4a21261940e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:17:50.419142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0d59df72-9e4a-4ef4-8fa2-fc147a5fff36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:50.483990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1278/2531] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:17:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:17:50.488529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e16da-e398-4bc1-9ee2-8cd0d542be3a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.559241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e16da-e398-4bc1-9ee2-8cd0d542be3a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.563906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75068f04-8c5d-4bfa-8c97-54cd6af196b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.564508 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75068f04-8c5d-4bfa-8c97-54cd6af196b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.564783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75068f04-8c5d-4bfa-8c97-54cd6af196b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.565049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75068f04-8c5d-4bfa-8c97-54cd6af196b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.565683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75068f04-8c5d-4bfa-8c97-54cd6af196b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.567599 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75068f04-8c5d-4bfa-8c97-54cd6af196b8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.594751 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1254/2532] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:17:50 2022] POST /identity/v3/projects => generated 357 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:17:50.598750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59df72-9e4a-4ef4-8fa2-fc147a5fff36 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.609546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59df72-9e4a-4ef4-8fa2-fc147a5fff36 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.664929 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20209e1-a3cb-4e07-b5d2-28a088755dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.665562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20209e1-a3cb-4e07-b5d2-28a088755dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.665843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20209e1-a3cb-4e07-b5d2-28a088755dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.666116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20209e1-a3cb-4e07-b5d2-28a088755dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.666816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20209e1-a3cb-4e07-b5d2-28a088755dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.668779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20209e1-a3cb-4e07-b5d2-28a088755dd7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.689555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1279/2533] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:17:50 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:17:50.693564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75068f04-8c5d-4bfa-8c97-54cd6af196b8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.704374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75068f04-8c5d-4bfa-8c97-54cd6af196b8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.708925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581b670-209e-4c5e-a57c-b860ca925fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.709513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581b670-209e-4c5e-a57c-b860ca925fef admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.709789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581b670-209e-4c5e-a57c-b860ca925fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.710086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581b670-209e-4c5e-a57c-b860ca925fef admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.710716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581b670-209e-4c5e-a57c-b860ca925fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.712339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581b670-209e-4c5e-a57c-b860ca925fef admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.723420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1255/2534] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:17:50.727574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20209e1-a3cb-4e07-b5d2-28a088755dd7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.737519 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20209e1-a3cb-4e07-b5d2-28a088755dd7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.742186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7db64-6b61-478b-8daa-c03d27904464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.742831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7db64-6b61-478b-8daa-c03d27904464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.743143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7db64-6b61-478b-8daa-c03d27904464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.743431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7db64-6b61-478b-8daa-c03d27904464 admin admin] PATH_INFO: `/v3/projects/3a3881d2243842029a6e32f1ad1089af/users/25c1579748bf445f901f06e00d4d5d66/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.758193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7db64-6b61-478b-8daa-c03d27904464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a3881d2243842029a6e32f1ad1089af, user_id=25c1579748bf445f901f06e00d4d5d66, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.760887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7db64-6b61-478b-8daa-c03d27904464 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.775046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1280/2535] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:17:50 2022] PUT /identity/v3/projects/3a3881d2243842029a6e32f1ad1089af/users/25c1579748bf445f901f06e00d4d5d66/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:17:50.779157 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581b670-209e-4c5e-a57c-b860ca925fef admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.789781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581b670-209e-4c5e-a57c-b860ca925fef admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.848023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d2e4d-ef35-4727-94df-b40ca83f0e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.848560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d2e4d-ef35-4727-94df-b40ca83f0e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.848827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d2e4d-ef35-4727-94df-b40ca83f0e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.849104 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d2e4d-ef35-4727-94df-b40ca83f0e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.849744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d2e4d-ef35-4727-94df-b40ca83f0e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.851376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d2e4d-ef35-4727-94df-b40ca83f0e0e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.861285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1256/2536] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:17:50.865799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7db64-6b61-478b-8daa-c03d27904464 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:50.876812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7db64-6b61-478b-8daa-c03d27904464 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:50.881905 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3fcc5-746f-432b-b0da-baf6912c9c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:50.882210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3fcc5-746f-432b-b0da-baf6912c9c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.882486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3fcc5-746f-432b-b0da-baf6912c9c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.882755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3fcc5-746f-432b-b0da-baf6912c9c6e admin admin] PATH_INFO: `/v3/projects/3a3881d2243842029a6e32f1ad1089af/users/25c1579748bf445f901f06e00d4d5d66/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.884910 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3fcc5-746f-432b-b0da-baf6912c9c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a3881d2243842029a6e32f1ad1089af, user_id=25c1579748bf445f901f06e00d4d5d66, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:50.887713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3fcc5-746f-432b-b0da-baf6912c9c6e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:50.900338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1281/2537] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:17:50 2022] PUT /identity/v3/projects/3a3881d2243842029a6e32f1ad1089af/users/25c1579748bf445f901f06e00d4d5d66/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:17:50.912584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d51b97-e4d5-407b-8f61-63511b7270af None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:50.912841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d51b97-e4d5-407b-8f61-63511b7270af None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:50.913035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d51b97-e4d5-407b-8f61-63511b7270af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:50.941883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-92d51b97-e4d5-407b-8f61-63511b7270af None None] MFA Rules not processed for user `25c1579748bf445f901f06e00d4d5d66`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:17:50.943115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-92d51b97-e4d5-407b-8f61-63511b7270af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:50.952788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:17:51.009750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1257/2538] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:17:50 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:17:51.021868 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3fcc5-746f-432b-b0da-baf6912c9c6e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:51.023620 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c6f3fcc5-746f-432b-b0da-baf6912c9c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:51.034894 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:17:51.082737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3fcc5-746f-432b-b0da-baf6912c9c6e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:51.086346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9127bf-8871-4733-a7a3-f96abc8a347d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:51.086938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9127bf-8871-4733-a7a3-f96abc8a347d service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:51.087244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9127bf-8871-4733-a7a3-f96abc8a347d service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:51.087512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9127bf-8871-4733-a7a3-f96abc8a347d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:51.096269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9127bf-8871-4733-a7a3-f96abc8a347d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:51.097674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9127bf-8871-4733-a7a3-f96abc8a347d service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:51.107113 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1282/2539] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:17:51 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:17:51.184811 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d51b97-e4d5-407b-8f61-63511b7270af None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:51.195768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d51b97-e4d5-407b-8f61-63511b7270af None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:51.199536 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97be78-9b05-41d1-9cf1-8a5262d81ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:51.200136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97be78-9b05-41d1-9cf1-8a5262d81ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:51.200442 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97be78-9b05-41d1-9cf1-8a5262d81ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:51.200716 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97be78-9b05-41d1-9cf1-8a5262d81ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:51.259134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97be78-9b05-41d1-9cf1-8a5262d81ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:51.260592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97be78-9b05-41d1-9cf1-8a5262d81ec0 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:51.269619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1258/2540] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:17:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:17:51.509993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9127bf-8871-4733-a7a3-f96abc8a347d service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:17:51.518855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9127bf-8871-4733-a7a3-f96abc8a347d service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:17:51.522636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771d7d3-2c52-421c-a124-917641fadea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:17:51.523280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771d7d3-2c52-421c-a124-917641fadea3 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:17:51.523560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771d7d3-2c52-421c-a124-917641fadea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:17:51.523830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771d7d3-2c52-421c-a124-917641fadea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:17:51.525544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7771d7d3-2c52-421c-a124-917641fadea3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:17:51.586316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771d7d3-2c52-421c-a124-917641fadea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:17:51.587778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771d7d3-2c52-421c-a124-917641fadea3 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:17:51.597214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1283/2541] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:17:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:01.265468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=52 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:01.593461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=50 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:02.728233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cf212-fed0-4091-9623-70ef168c24b0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:02.728782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cf212-fed0-4091-9623-70ef168c24b0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:02.728782 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cf212-fed0-4091-9623-70ef168c24b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:02.748471 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e22cf212-fed0-4091-9623-70ef168c24b0 None None] MFA Rules not processed for user `c2060b33d0a24e01a96ab3fb02b4b045`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:02.750264 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e22cf212-fed0-4091-9623-70ef168c24b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:02.757953 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1259/2542] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:03.084399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771d7d3-2c52-421c-a124-917641fadea3 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:03.085686 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7771d7d3-2c52-421c-a124-917641fadea3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:03.143341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771d7d3-2c52-421c-a124-917641fadea3 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:03.146885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e689e-57c9-4c16-969e-041663c3cdbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:03.147515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e689e-57c9-4c16-969e-041663c3cdbc service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:03.147792 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e689e-57c9-4c16-969e-041663c3cdbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:03.148073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e689e-57c9-4c16-969e-041663c3cdbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:03.149777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a25e689e-57c9-4c16-969e-041663c3cdbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:03.159760 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e689e-57c9-4c16-969e-041663c3cdbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:03.161164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e689e-57c9-4c16-969e-041663c3cdbc service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:03.169318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1284/2543] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:03.656850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cf212-fed0-4091-9623-70ef168c24b0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:03.658286 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e22cf212-fed0-4091-9623-70ef168c24b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:03.733137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cf212-fed0-4091-9623-70ef168c24b0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:03.738070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee4766-4599-4b13-96fe-89312706681a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:03.738681 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee4766-4599-4b13-96fe-89312706681a service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:03.738983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee4766-4599-4b13-96fe-89312706681a service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:03.739262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee4766-4599-4b13-96fe-89312706681a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:03.740959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cdee4766-4599-4b13-96fe-89312706681a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:03.753332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee4766-4599-4b13-96fe-89312706681a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:03.755289 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee4766-4599-4b13-96fe-89312706681a service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:03.766804 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1260/2544] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:05.718481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e689e-57c9-4c16-969e-041663c3cdbc service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:05.727613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e689e-57c9-4c16-969e-041663c3cdbc service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:05.731331 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5435230-625b-455b-b000-6ec64f2c9469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:05.731937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5435230-625b-455b-b000-6ec64f2c9469 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:05.732214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5435230-625b-455b-b000-6ec64f2c9469 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:05.732482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5435230-625b-455b-b000-6ec64f2c9469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:05.734788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c5435230-625b-455b-b000-6ec64f2c9469 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:05.745297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5435230-625b-455b-b000-6ec64f2c9469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:05.747028 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5435230-625b-455b-b000-6ec64f2c9469 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:05.757385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1285/2545] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:12.996047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee4766-4599-4b13-96fe-89312706681a service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:13.064635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee4766-4599-4b13-96fe-89312706681a service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:13.069093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:13.069711 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:13.069984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:13.070250 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] PATH_INFO: `/v3/users/68fe760a598b46e6a7a4a21261940e32` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:13.071398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68fe760a598b46e6a7a4a21261940e32)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:13.073301 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:13.119687 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68fe760a598b46e6a7a4a21261940e32'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.131381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68fe760a598b46e6a7a4a21261940e32'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.138092 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68fe760a598b46e6a7a4a21261940e32'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.138769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68fe760a598b46e6a7a4a21261940e32'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.146668 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68fe760a598b46e6a7a4a21261940e32'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.152732 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1261/2546] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:18:12 2022] DELETE /identity/v3/users/68fe760a598b46e6a7a4a21261940e32 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:13.158151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2429c-13ad-41e1-a0dc-e8d1513ed9d3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:13.158666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2429c-13ad-41e1-a0dc-e8d1513ed9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:13.158789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2429c-13ad-41e1-a0dc-e8d1513ed9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:13.177641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bae2429c-13ad-41e1-a0dc-e8d1513ed9d3 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:13.178837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bae2429c-13ad-41e1-a0dc-e8d1513ed9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:13.238680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1286/2547] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:18:13.248666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:13.250227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:13.255579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:13.315415 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da4405-aee3-44e8-8d72-e4479c1bec45 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:13.319176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ceb5ca-5ea8-4a91-b819-9c51a98da10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:13.319786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceb5ca-5ea8-4a91-b819-9c51a98da10b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:13.320064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceb5ca-5ea8-4a91-b819-9c51a98da10b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:13.320335 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceb5ca-5ea8-4a91-b819-9c51a98da10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:13.330784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ceb5ca-5ea8-4a91-b819-9c51a98da10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:13.332697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ceb5ca-5ea8-4a91-b819-9c51a98da10b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:13.347750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1262/2548] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:13.629413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2429c-13ad-41e1-a0dc-e8d1513ed9d3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:13.630846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bae2429c-13ad-41e1-a0dc-e8d1513ed9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:13.646540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2429c-13ad-41e1-a0dc-e8d1513ed9d3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:13.652103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:13.653221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:13.653367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:13.653817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] PATH_INFO: `/v3/projects/7b3862a3adf5486e96bc9292f6296e38` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:13.655197 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3862a3adf5486e96bc9292f6296e38)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:13.657025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:13.711196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3862a3adf5486e96bc9292f6296e38', 'request_id': 'req-244c90e3-cd9f-4eff-8f1f-f352e4402372'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.718007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3862a3adf5486e96bc9292f6296e38', 'request_id': 'req-244c90e3-cd9f-4eff-8f1f-f352e4402372'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.723996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3862a3adf5486e96bc9292f6296e38', 'request_id': 'req-244c90e3-cd9f-4eff-8f1f-f352e4402372'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.738140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] The token cache is being invalidate because project 7b3862a3adf5486e96bc9292f6296e38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:18:13.738426 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.739080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.740570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1287/2549] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:18:13 2022] DELETE /identity/v3/projects/7b3862a3adf5486e96bc9292f6296e38 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:13.744586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ceb5ca-5ea8-4a91-b819-9c51a98da10b service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:13.754812 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ceb5ca-5ea8-4a91-b819-9c51a98da10b service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:13.810186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834f54e-8f36-4a66-ab3a-20299ff52870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:13.810816 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834f54e-8f36-4a66-ab3a-20299ff52870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:13.811124 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834f54e-8f36-4a66-ab3a-20299ff52870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:13.812036 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834f54e-8f36-4a66-ab3a-20299ff52870 admin admin] PATH_INFO: `/v3/users/25c1579748bf445f901f06e00d4d5d66` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:13.813149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834f54e-8f36-4a66-ab3a-20299ff52870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c1579748bf445f901f06e00d4d5d66)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:13.815131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834f54e-8f36-4a66-ab3a-20299ff52870 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:13.861531 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7834f54e-8f36-4a66-ab3a-20299ff52870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c1579748bf445f901f06e00d4d5d66'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.874631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7834f54e-8f36-4a66-ab3a-20299ff52870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c1579748bf445f901f06e00d4d5d66'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.881633 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7834f54e-8f36-4a66-ab3a-20299ff52870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c1579748bf445f901f06e00d4d5d66'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.882336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7834f54e-8f36-4a66-ab3a-20299ff52870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c1579748bf445f901f06e00d4d5d66'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.889760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7834f54e-8f36-4a66-ab3a-20299ff52870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c1579748bf445f901f06e00d4d5d66'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:13.896146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1263/2550] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:18:13 2022] DELETE /identity/v3/users/25c1579748bf445f901f06e00d4d5d66 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:14.194353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:14.196018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:14.209949 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c90e3-cd9f-4eff-8f1f-f352e4402372 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:14.219945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:14.271600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:14.272143 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:14.272413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:14.272754 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] PATH_INFO: `/v3/projects/3a3881d2243842029a6e32f1ad1089af` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:14.273803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a3881d2243842029a6e32f1ad1089af)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:14.275778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:14.323950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a3881d2243842029a6e32f1ad1089af', 'request_id': 'req-76e32add-c34b-45f8-b698-09aeb8613111'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:14.330598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a3881d2243842029a6e32f1ad1089af', 'request_id': 'req-76e32add-c34b-45f8-b698-09aeb8613111'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:14.336593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a3881d2243842029a6e32f1ad1089af', 'request_id': 'req-76e32add-c34b-45f8-b698-09aeb8613111'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:14.348984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] The token cache is being invalidate because project 3a3881d2243842029a6e32f1ad1089af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:18:14.349278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:14.350115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-76e32add-c34b-45f8-b698-09aeb8613111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:14.351627 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1288/2551] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:18:14 2022] DELETE /identity/v3/projects/3a3881d2243842029a6e32f1ad1089af => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:18:14.357615 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a99a46-df86-407d-b1c0-b6a6da5d5a8d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:14.358110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a99a46-df86-407d-b1c0-b6a6da5d5a8d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:14.358819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a99a46-df86-407d-b1c0-b6a6da5d5a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:14.378210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-76a99a46-df86-407d-b1c0-b6a6da5d5a8d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:14.379448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-76a99a46-df86-407d-b1c0-b6a6da5d5a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:14.397689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:14.442199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1264/2552] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:18:14.450123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144762e-6e9a-47c5-94a7-74283f39a6b9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:14.450411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144762e-6e9a-47c5-94a7-74283f39a6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:14.450999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144762e-6e9a-47c5-94a7-74283f39a6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:14.472010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5144762e-6e9a-47c5-94a7-74283f39a6b9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:14.473255 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5144762e-6e9a-47c5-94a7-74283f39a6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:14.481278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1289/2553] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:14.485780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a99a46-df86-407d-b1c0-b6a6da5d5a8d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:14.495491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a99a46-df86-407d-b1c0-b6a6da5d5a8d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:14.500267 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2e1d1-1435-4178-bd12-618ab0e42cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:14.500867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2e1d1-1435-4178-bd12-618ab0e42cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:14.501143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2e1d1-1435-4178-bd12-618ab0e42cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:14.501409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2e1d1-1435-4178-bd12-618ab0e42cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:14.502045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2e1d1-1435-4178-bd12-618ab0e42cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:18:14.502347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2e1d1-1435-4178-bd12-618ab0e42cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:14.503984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2e1d1-1435-4178-bd12-618ab0e42cfe admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:14.516537 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1265/2554] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:18:14.520883 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144762e-6e9a-47c5-94a7-74283f39a6b9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:14.531669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144762e-6e9a-47c5-94a7-74283f39a6b9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:14.536681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af6e32-5a58-4b89-a7fd-bda5eba4a935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:14.537304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af6e32-5a58-4b89-a7fd-bda5eba4a935 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:14.537587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af6e32-5a58-4b89-a7fd-bda5eba4a935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:14.538394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af6e32-5a58-4b89-a7fd-bda5eba4a935 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:14.538518 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af6e32-5a58-4b89-a7fd-bda5eba4a935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:14.540440 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af6e32-5a58-4b89-a7fd-bda5eba4a935 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:14.569596 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1290/2555] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:18:14 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:18:14.573919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2e1d1-1435-4178-bd12-618ab0e42cfe admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:14.584449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2e1d1-1435-4178-bd12-618ab0e42cfe admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:14.640319 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55d3de-4425-40bb-92c6-c35594b996c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:14.640927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55d3de-4425-40bb-92c6-c35594b996c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:14.641209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55d3de-4425-40bb-92c6-c35594b996c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:14.641480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55d3de-4425-40bb-92c6-c35594b996c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:14.642110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55d3de-4425-40bb-92c6-c35594b996c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:14.644091 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55d3de-4425-40bb-92c6-c35594b996c7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:14.666583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1266/2556] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:18:14 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:18:14.670481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af6e32-5a58-4b89-a7fd-bda5eba4a935 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:14.681383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af6e32-5a58-4b89-a7fd-bda5eba4a935 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:14.685986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4728cce-f93d-4c29-a4cb-6d501385e014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:14.686583 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4728cce-f93d-4c29-a4cb-6d501385e014 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:14.686864 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4728cce-f93d-4c29-a4cb-6d501385e014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:14.687171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4728cce-f93d-4c29-a4cb-6d501385e014 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:14.687811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4728cce-f93d-4c29-a4cb-6d501385e014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:14.689387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4728cce-f93d-4c29-a4cb-6d501385e014 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:14.699235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1291/2557] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:14.703624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55d3de-4425-40bb-92c6-c35594b996c7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:14.713510 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55d3de-4425-40bb-92c6-c35594b996c7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:14.718683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837312e-0dc1-4484-947b-42e436bc7162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:14.719298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837312e-0dc1-4484-947b-42e436bc7162 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:14.719576 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837312e-0dc1-4484-947b-42e436bc7162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:14.719847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837312e-0dc1-4484-947b-42e436bc7162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:14.720480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837312e-0dc1-4484-947b-42e436bc7162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:14.722060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837312e-0dc1-4484-947b-42e436bc7162 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:14.732477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1267/2558] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:14.736609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4728cce-f93d-4c29-a4cb-6d501385e014 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:14.746744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4728cce-f93d-4c29-a4cb-6d501385e014 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:14.751475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac956e74-c558-4be7-bd98-33f7663da5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:14.752134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac956e74-c558-4be7-bd98-33f7663da5e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:14.752417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac956e74-c558-4be7-bd98-33f7663da5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:14.752691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac956e74-c558-4be7-bd98-33f7663da5e8 admin admin] PATH_INFO: `/v3/projects/be8d41d150dd49538dee8c656928e05b/users/d0ecac93f4784e37bbb1858c2a16dcfa/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:14.768605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac956e74-c558-4be7-bd98-33f7663da5e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be8d41d150dd49538dee8c656928e05b, user_id=d0ecac93f4784e37bbb1858c2a16dcfa, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:14.771275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac956e74-c558-4be7-bd98-33f7663da5e8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:14.784806 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1292/2559] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:18:14 2022] PUT /identity/v3/projects/be8d41d150dd49538dee8c656928e05b/users/d0ecac93f4784e37bbb1858c2a16dcfa/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:18:14.794983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45798df-d0a3-4683-a4e0-23b06cd1bc75 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:14.795272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45798df-d0a3-4683-a4e0-23b06cd1bc75 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:14.795544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45798df-d0a3-4683-a4e0-23b06cd1bc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:14.823445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c45798df-d0a3-4683-a4e0-23b06cd1bc75 None None] MFA Rules not processed for user `d0ecac93f4784e37bbb1858c2a16dcfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:14.824644 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c45798df-d0a3-4683-a4e0-23b06cd1bc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:14.888874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1268/2560] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:14.893959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac956e74-c558-4be7-bd98-33f7663da5e8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:14.937305 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:14.959149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac956e74-c558-4be7-bd98-33f7663da5e8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:14.963972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28893f16-6c5b-441e-8fad-b707a486d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:14.964618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893f16-6c5b-441e-8fad-b707a486d6ac admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:14.964899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893f16-6c5b-441e-8fad-b707a486d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:14.965170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893f16-6c5b-441e-8fad-b707a486d6ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:14.965942 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28893f16-6c5b-441e-8fad-b707a486d6ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:14.967896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28893f16-6c5b-441e-8fad-b707a486d6ac admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:14.996002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1293/2561] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:18:14 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:18:15.000528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45798df-d0a3-4683-a4e0-23b06cd1bc75 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:15.011647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45798df-d0a3-4683-a4e0-23b06cd1bc75 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:15.029493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:15.074414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a7bbd-dd7d-41af-b3c4-9b08efd10c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:15.075274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7bbd-dd7d-41af-b3c4-9b08efd10c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:15.075559 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7bbd-dd7d-41af-b3c4-9b08efd10c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:15.075836 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7bbd-dd7d-41af-b3c4-9b08efd10c01 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:15.076539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a7bbd-dd7d-41af-b3c4-9b08efd10c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:15.079265 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a7bbd-dd7d-41af-b3c4-9b08efd10c01 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:15.106867 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1269/2562] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:18:14 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:18:15.108399 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28893f16-6c5b-441e-8fad-b707a486d6ac admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:15.119967 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28893f16-6c5b-441e-8fad-b707a486d6ac admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:15.125378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b6a5d-190d-4bb3-90ee-210c4399d012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:15.126128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b6a5d-190d-4bb3-90ee-210c4399d012 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:15.126414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b6a5d-190d-4bb3-90ee-210c4399d012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:15.126693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b6a5d-190d-4bb3-90ee-210c4399d012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:15.127431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b6a5d-190d-4bb3-90ee-210c4399d012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:15.129066 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b6a5d-190d-4bb3-90ee-210c4399d012 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:15.141145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1294/2563] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:15.145818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a7bbd-dd7d-41af-b3c4-9b08efd10c01 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:15.157516 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a7bbd-dd7d-41af-b3c4-9b08efd10c01 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:15.162424 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099770e-85aa-438a-b843-ddcce5cd39e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:15.163131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099770e-85aa-438a-b843-ddcce5cd39e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:15.163416 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099770e-85aa-438a-b843-ddcce5cd39e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:15.163748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099770e-85aa-438a-b843-ddcce5cd39e0 admin admin] PATH_INFO: `/v3/projects/e70d7cc2bc9f4b9d94143f9828138fad/users/e0e98b552b3e4208b5bfdc023ef7fcd6/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:15.180001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099770e-85aa-438a-b843-ddcce5cd39e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70d7cc2bc9f4b9d94143f9828138fad, user_id=e0e98b552b3e4208b5bfdc023ef7fcd6, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:15.182664 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099770e-85aa-438a-b843-ddcce5cd39e0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:15.207475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1270/2564] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:18:15 2022] PUT /identity/v3/projects/e70d7cc2bc9f4b9d94143f9828138fad/users/e0e98b552b3e4208b5bfdc023ef7fcd6/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:15.212414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b6a5d-190d-4bb3-90ee-210c4399d012 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:15.227240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b6a5d-190d-4bb3-90ee-210c4399d012 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:15.301772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c10dc-0d07-4b00-b824-1d025ff33964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:15.302410 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c10dc-0d07-4b00-b824-1d025ff33964 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:15.302691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c10dc-0d07-4b00-b824-1d025ff33964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:15.304728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c10dc-0d07-4b00-b824-1d025ff33964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:15.305713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c10dc-0d07-4b00-b824-1d025ff33964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:15.307055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c10dc-0d07-4b00-b824-1d025ff33964 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:15.321544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1295/2565] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:15.322865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099770e-85aa-438a-b843-ddcce5cd39e0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:15.335336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099770e-85aa-438a-b843-ddcce5cd39e0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:15.340225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3ffac-8a4a-4795-8c6b-28f8879b385d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:15.340999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3ffac-8a4a-4795-8c6b-28f8879b385d admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:15.341294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3ffac-8a4a-4795-8c6b-28f8879b385d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:15.341640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3ffac-8a4a-4795-8c6b-28f8879b385d admin admin] PATH_INFO: `/v3/projects/e70d7cc2bc9f4b9d94143f9828138fad/users/e0e98b552b3e4208b5bfdc023ef7fcd6/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:15.343750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3ffac-8a4a-4795-8c6b-28f8879b385d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70d7cc2bc9f4b9d94143f9828138fad, user_id=e0e98b552b3e4208b5bfdc023ef7fcd6, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:15.346822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3ffac-8a4a-4795-8c6b-28f8879b385d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:15.371065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1271/2566] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:18:15 2022] PUT /identity/v3/projects/e70d7cc2bc9f4b9d94143f9828138fad/users/e0e98b552b3e4208b5bfdc023ef7fcd6/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:15.374662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a00883-2f7f-4631-acf9-30ed2f8a01a7 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:15.374662 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a00883-2f7f-4631-acf9-30ed2f8a01a7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:15.375033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a00883-2f7f-4631-acf9-30ed2f8a01a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:15.407395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-23a00883-2f7f-4631-acf9-30ed2f8a01a7 None None] MFA Rules not processed for user `e0e98b552b3e4208b5bfdc023ef7fcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:15.408010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-23a00883-2f7f-4631-acf9-30ed2f8a01a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:15.500765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1296/2567] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:15.513602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3ffac-8a4a-4795-8c6b-28f8879b385d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:15.516877 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-60b3ffac-8a4a-4795-8c6b-28f8879b385d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:15.603789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3ffac-8a4a-4795-8c6b-28f8879b385d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:15.608640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05540e52-1532-46da-8cd0-3256689fdd71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:15.610011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05540e52-1532-46da-8cd0-3256689fdd71 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:15.610011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05540e52-1532-46da-8cd0-3256689fdd71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:15.610193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05540e52-1532-46da-8cd0-3256689fdd71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:15.710307 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05540e52-1532-46da-8cd0-3256689fdd71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:15.711831 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05540e52-1532-46da-8cd0-3256689fdd71 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:15.722379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1272/2568] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:18:23.526104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a00883-2f7f-4631-acf9-30ed2f8a01a7 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:23.527540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-23a00883-2f7f-4631-acf9-30ed2f8a01a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:23.539927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a00883-2f7f-4631-acf9-30ed2f8a01a7 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:23.543508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee94558-2af7-4a25-95b3-78df93b32e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:23.544108 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee94558-2af7-4a25-95b3-78df93b32e7a service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:23.544387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee94558-2af7-4a25-95b3-78df93b32e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:23.544658 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee94558-2af7-4a25-95b3-78df93b32e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:23.554368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee94558-2af7-4a25-95b3-78df93b32e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:23.555772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee94558-2af7-4a25-95b3-78df93b32e7a service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:23.565979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1297/2569] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:24.176467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05540e52-1532-46da-8cd0-3256689fdd71 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:24.178198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-05540e52-1532-46da-8cd0-3256689fdd71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:24.214704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:24.259170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05540e52-1532-46da-8cd0-3256689fdd71 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:24.265279 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d7895-91a5-4f68-9b5c-185b2b06b4a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:24.265891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d7895-91a5-4f68-9b5c-185b2b06b4a8 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:24.266171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d7895-91a5-4f68-9b5c-185b2b06b4a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:24.266443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d7895-91a5-4f68-9b5c-185b2b06b4a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:24.268242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0c9d7895-91a5-4f68-9b5c-185b2b06b4a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:24.281361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d7895-91a5-4f68-9b5c-185b2b06b4a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:24.282984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d7895-91a5-4f68-9b5c-185b2b06b4a8 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:24.293550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1273/2570] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:30.222118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee94558-2af7-4a25-95b3-78df93b32e7a service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:30.223715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6ee94558-2af7-4a25-95b3-78df93b32e7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:30.250769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:30.290211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee94558-2af7-4a25-95b3-78df93b32e7a service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:30.293812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397b3bb-6fc7-4c4f-9a82-276d4bfa45c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:30.294444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397b3bb-6fc7-4c4f-9a82-276d4bfa45c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:30.294723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397b3bb-6fc7-4c4f-9a82-276d4bfa45c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:30.295015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397b3bb-6fc7-4c4f-9a82-276d4bfa45c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:30.297079 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d397b3bb-6fc7-4c4f-9a82-276d4bfa45c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:30.366224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397b3bb-6fc7-4c4f-9a82-276d4bfa45c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:30.367682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397b3bb-6fc7-4c4f-9a82-276d4bfa45c4 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:30.377479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1298/2571] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:30.764071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d7895-91a5-4f68-9b5c-185b2b06b4a8 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:30.775317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d7895-91a5-4f68-9b5c-185b2b06b4a8 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:30.779108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca56cb-5910-430b-a058-385a230d6c48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:30.779787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca56cb-5910-430b-a058-385a230d6c48 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:30.780093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca56cb-5910-430b-a058-385a230d6c48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:30.780369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca56cb-5910-430b-a058-385a230d6c48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:30.782129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-73ca56cb-5910-430b-a058-385a230d6c48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:30.850126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca56cb-5910-430b-a058-385a230d6c48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:30.851914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca56cb-5910-430b-a058-385a230d6c48 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:30.862737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1274/2572] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:34.186968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397b3bb-6fc7-4c4f-9a82-276d4bfa45c4 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:34.188478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d397b3bb-6fc7-4c4f-9a82-276d4bfa45c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:34.203669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397b3bb-6fc7-4c4f-9a82-276d4bfa45c4 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:34.209040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4aadf-78fe-4420-92e1-3a4cd7790e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:34.209718 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4aadf-78fe-4420-92e1-3a4cd7790e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:34.210001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4aadf-78fe-4420-92e1-3a4cd7790e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:34.210276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4aadf-78fe-4420-92e1-3a4cd7790e6d admin admin] PATH_INFO: `/v3/users/5feea3b3b7e4401ea7db242cb60fdfcc` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:34.227111 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4aadf-78fe-4420-92e1-3a4cd7790e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5feea3b3b7e4401ea7db242cb60fdfcc)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:34.229353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4aadf-78fe-4420-92e1-3a4cd7790e6d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:34.284789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-10c4aadf-78fe-4420-92e1-3a4cd7790e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5feea3b3b7e4401ea7db242cb60fdfcc'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:34.298067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-10c4aadf-78fe-4420-92e1-3a4cd7790e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5feea3b3b7e4401ea7db242cb60fdfcc'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:34.306175 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-10c4aadf-78fe-4420-92e1-3a4cd7790e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5feea3b3b7e4401ea7db242cb60fdfcc'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:34.316394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-10c4aadf-78fe-4420-92e1-3a4cd7790e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5feea3b3b7e4401ea7db242cb60fdfcc'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:34.317114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-10c4aadf-78fe-4420-92e1-3a4cd7790e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5feea3b3b7e4401ea7db242cb60fdfcc'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:34.323986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1299/2573] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:18:34 2022] DELETE /identity/v3/users/5feea3b3b7e4401ea7db242cb60fdfcc => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:34.500440 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca56cb-5910-430b-a058-385a230d6c48 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:34.510736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca56cb-5910-430b-a058-385a230d6c48 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:34.571400 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:34.572030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:34.572310 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:34.572583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] PATH_INFO: `/v3/projects/a6e3ace80a4a43b2b187060d6eca8538` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:34.583959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e3ace80a4a43b2b187060d6eca8538)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:34.585906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:34.625719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:34.633318 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e3ace80a4a43b2b187060d6eca8538', 'request_id': 'req-f362a976-be36-44d6-b338-f964991c19d8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:34.638994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e3ace80a4a43b2b187060d6eca8538', 'request_id': 'req-f362a976-be36-44d6-b338-f964991c19d8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:34.646227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e3ace80a4a43b2b187060d6eca8538', 'request_id': 'req-f362a976-be36-44d6-b338-f964991c19d8'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:34.659046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] The token cache is being invalidate because project a6e3ace80a4a43b2b187060d6eca8538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:18:34.659337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:34.659963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f362a976-be36-44d6-b338-f964991c19d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:34.661492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1275/2574] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:18:34 2022] DELETE /identity/v3/projects/a6e3ace80a4a43b2b187060d6eca8538 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:18:34.666560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a712f5b-dc63-4b63-874c-68ca1c44d988 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:34.666848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a712f5b-dc63-4b63-874c-68ca1c44d988 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:34.667172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a712f5b-dc63-4b63-874c-68ca1c44d988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:34.687937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-9a712f5b-dc63-4b63-874c-68ca1c44d988 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:34.689138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9a712f5b-dc63-4b63-874c-68ca1c44d988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:34.717144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:34.750678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1300/2575] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:34.758852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6556de9-11c7-4f8b-a585-87c398db12d9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:34.759077 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6556de9-11c7-4f8b-a585-87c398db12d9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:34.759362 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6556de9-11c7-4f8b-a585-87c398db12d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:34.779691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a6556de9-11c7-4f8b-a585-87c398db12d9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:34.781002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a6556de9-11c7-4f8b-a585-87c398db12d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:34.789379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1276/2576] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:34.793726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a712f5b-dc63-4b63-874c-68ca1c44d988 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:34.804834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a712f5b-dc63-4b63-874c-68ca1c44d988 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:34.809531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a70b3-7ce6-4550-966a-2f7f475ee678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:34.810273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a70b3-7ce6-4550-966a-2f7f475ee678 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:34.810552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a70b3-7ce6-4550-966a-2f7f475ee678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:34.810826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a70b3-7ce6-4550-966a-2f7f475ee678 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:34.812159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a70b3-7ce6-4550-966a-2f7f475ee678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:18:34.812456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a70b3-7ce6-4550-966a-2f7f475ee678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:34.814055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a70b3-7ce6-4550-966a-2f7f475ee678 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:34.825839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1301/2577] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:18:34.830315 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6556de9-11c7-4f8b-a585-87c398db12d9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:34.839658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6556de9-11c7-4f8b-a585-87c398db12d9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:34.844488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a7549-3c4c-485f-a251-2a59f60e7ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:34.845081 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a7549-3c4c-485f-a251-2a59f60e7ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:34.845363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a7549-3c4c-485f-a251-2a59f60e7ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:34.845636 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a7549-3c4c-485f-a251-2a59f60e7ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:34.846274 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a7549-3c4c-485f-a251-2a59f60e7ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:34.848208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a7549-3c4c-485f-a251-2a59f60e7ff4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:34.882706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1277/2578] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:18:34 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:18:34.884108 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a70b3-7ce6-4550-966a-2f7f475ee678 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:34.894560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a70b3-7ce6-4550-966a-2f7f475ee678 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:34.955322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f62c37-05e7-457c-9a42-40454c53945c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:34.955935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f62c37-05e7-457c-9a42-40454c53945c admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:34.956215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f62c37-05e7-457c-9a42-40454c53945c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:34.956486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f62c37-05e7-457c-9a42-40454c53945c admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:34.957115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f62c37-05e7-457c-9a42-40454c53945c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:34.959033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f62c37-05e7-457c-9a42-40454c53945c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:34.982848 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1302/2579] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:18:34 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:18:34.987956 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a7549-3c4c-485f-a251-2a59f60e7ff4 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:34.999477 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a7549-3c4c-485f-a251-2a59f60e7ff4 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:35.004137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b206231-8753-43f3-9f3a-d3f8b8d6b9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:35.004741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b206231-8753-43f3-9f3a-d3f8b8d6b9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:35.005022 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b206231-8753-43f3-9f3a-d3f8b8d6b9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:35.005320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b206231-8753-43f3-9f3a-d3f8b8d6b9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:35.005958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b206231-8753-43f3-9f3a-d3f8b8d6b9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:35.007584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b206231-8753-43f3-9f3a-d3f8b8d6b9b1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:35.017835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1278/2580] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:35.021946 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f62c37-05e7-457c-9a42-40454c53945c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:35.033160 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f62c37-05e7-457c-9a42-40454c53945c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:35.039346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fa34f-4f8d-4e4e-8194-2382b54819f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:35.040073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fa34f-4f8d-4e4e-8194-2382b54819f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:35.040352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fa34f-4f8d-4e4e-8194-2382b54819f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:35.040665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fa34f-4f8d-4e4e-8194-2382b54819f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:35.041593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fa34f-4f8d-4e4e-8194-2382b54819f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:35.043497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fa34f-4f8d-4e4e-8194-2382b54819f5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:35.056242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1303/2581] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:35.060494 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b206231-8753-43f3-9f3a-d3f8b8d6b9b1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:35.071418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b206231-8753-43f3-9f3a-d3f8b8d6b9b1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:35.075997 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b7ede-0afc-468f-b762-2f9a1381abff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:35.076649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b7ede-0afc-468f-b762-2f9a1381abff admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:35.076926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b7ede-0afc-468f-b762-2f9a1381abff admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:35.077201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b7ede-0afc-468f-b762-2f9a1381abff admin admin] PATH_INFO: `/v3/projects/8ba8c6057ca64042aed56a34c4b867df/users/8c2fd4d5ef9e4568851097bb4a6c01f9/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:35.092178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b7ede-0afc-468f-b762-2f9a1381abff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba8c6057ca64042aed56a34c4b867df, user_id=8c2fd4d5ef9e4568851097bb4a6c01f9, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:35.095412 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b7ede-0afc-468f-b762-2f9a1381abff admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:35.119187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1279/2582] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:18:35 2022] PUT /identity/v3/projects/8ba8c6057ca64042aed56a34c4b867df/users/8c2fd4d5ef9e4568851097bb4a6c01f9/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:35.124429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b806-f74e-4d81-a626-ff5a7cda517e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:35.125183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b806-f74e-4d81-a626-ff5a7cda517e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:35.125183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b806-f74e-4d81-a626-ff5a7cda517e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:35.144279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e8f2b806-f74e-4d81-a626-ff5a7cda517e None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:35.145509 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e8f2b806-f74e-4d81-a626-ff5a7cda517e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:35.218679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1304/2583] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:35.227919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b7ede-0afc-468f-b762-2f9a1381abff admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:35.229721 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-362b7ede-0afc-468f-b762-2f9a1381abff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:35.289562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b7ede-0afc-468f-b762-2f9a1381abff admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:35.293136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c040b-fbf5-4a05-81ae-e0ae408c09aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:35.293745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c040b-fbf5-4a05-81ae-e0ae408c09aa service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:35.294024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c040b-fbf5-4a05-81ae-e0ae408c09aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:35.294295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c040b-fbf5-4a05-81ae-e0ae408c09aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:35.305713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c040b-fbf5-4a05-81ae-e0ae408c09aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:35.306829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c040b-fbf5-4a05-81ae-e0ae408c09aa service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:35.317443 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1280/2584] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:36.195966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af6a8e-4c03-455a-995e-e9a2b78f813c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:36.196258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af6a8e-4c03-455a-995e-e9a2b78f813c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:36.196534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af6a8e-4c03-455a-995e-e9a2b78f813c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:36.225489 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-36af6a8e-4c03-455a-995e-e9a2b78f813c None None] MFA Rules not processed for user `8c2fd4d5ef9e4568851097bb4a6c01f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:36.226770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-36af6a8e-4c03-455a-995e-e9a2b78f813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:36.260131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:36.297893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1305/2585] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:36 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:36.302772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c040b-fbf5-4a05-81ae-e0ae408c09aa service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:36.312796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c040b-fbf5-4a05-81ae-e0ae408c09aa service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:36.317423 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ab18d-aebf-47a3-a058-c066d973fe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:36.318033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ab18d-aebf-47a3-a058-c066d973fe21 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:36.318317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ab18d-aebf-47a3-a058-c066d973fe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:36.318587 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ab18d-aebf-47a3-a058-c066d973fe21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:36.319248 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ab18d-aebf-47a3-a058-c066d973fe21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:36.321140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ab18d-aebf-47a3-a058-c066d973fe21 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:36.349601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1281/2586] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:18:36 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:18:36.354121 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af6a8e-4c03-455a-995e-e9a2b78f813c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:36.366023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af6a8e-4c03-455a-995e-e9a2b78f813c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:36.424528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cbe46-e460-48c8-b832-de9ec938d97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:36.425149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cbe46-e460-48c8-b832-de9ec938d97e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:36.425429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cbe46-e460-48c8-b832-de9ec938d97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:36.425704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cbe46-e460-48c8-b832-de9ec938d97e admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:36.426333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cbe46-e460-48c8-b832-de9ec938d97e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:36.428294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cbe46-e460-48c8-b832-de9ec938d97e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:36.450446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1306/2587] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:18:36 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:18:36.455724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ab18d-aebf-47a3-a058-c066d973fe21 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:36.475678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ab18d-aebf-47a3-a058-c066d973fe21 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:36.480528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d9232-2fc6-4e73-bff2-e38cf6c1fa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:36.481136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d9232-2fc6-4e73-bff2-e38cf6c1fa5f admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:36.481418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d9232-2fc6-4e73-bff2-e38cf6c1fa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:36.481692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d9232-2fc6-4e73-bff2-e38cf6c1fa5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:36.482338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d9232-2fc6-4e73-bff2-e38cf6c1fa5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:36.484474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d9232-2fc6-4e73-bff2-e38cf6c1fa5f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:36.495671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1282/2588] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:18:36.500603 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cbe46-e460-48c8-b832-de9ec938d97e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:36.510856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cbe46-e460-48c8-b832-de9ec938d97e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:36.515541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297d288-ec6f-4d4a-b407-ad3b5975ecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:36.516192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297d288-ec6f-4d4a-b407-ad3b5975ecc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:36.516471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297d288-ec6f-4d4a-b407-ad3b5975ecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:36.516743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297d288-ec6f-4d4a-b407-ad3b5975ecc1 admin admin] PATH_INFO: `/v3/projects/475af14af9fa44ee981907bde05963dc/users/289f38f1ad73467484f7d1710669ceab/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:36.533944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297d288-ec6f-4d4a-b407-ad3b5975ecc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=475af14af9fa44ee981907bde05963dc, user_id=289f38f1ad73467484f7d1710669ceab, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:36.536782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297d288-ec6f-4d4a-b407-ad3b5975ecc1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:36.550147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1307/2589] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:18:36 2022] PUT /identity/v3/projects/475af14af9fa44ee981907bde05963dc/users/289f38f1ad73467484f7d1710669ceab/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:18:36.554534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d9232-2fc6-4e73-bff2-e38cf6c1fa5f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:36.564600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d9232-2fc6-4e73-bff2-e38cf6c1fa5f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:36.596285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:36.624753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c40b7-5ac5-4358-9e52-8a492f878531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:36.625353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c40b7-5ac5-4358-9e52-8a492f878531 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:36.625629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c40b7-5ac5-4358-9e52-8a492f878531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:36.625898 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c40b7-5ac5-4358-9e52-8a492f878531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:36.626532 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c40b7-5ac5-4358-9e52-8a492f878531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:36.628156 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c40b7-5ac5-4358-9e52-8a492f878531 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:36.637791 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1283/2590] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:36.641928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297d288-ec6f-4d4a-b407-ad3b5975ecc1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:36.652199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297d288-ec6f-4d4a-b407-ad3b5975ecc1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:36.657371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf4f0a-335d-4b5d-a16f-8676aa8bd730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:36.658011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf4f0a-335d-4b5d-a16f-8676aa8bd730 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:36.658287 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf4f0a-335d-4b5d-a16f-8676aa8bd730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:36.658562 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf4f0a-335d-4b5d-a16f-8676aa8bd730 admin admin] PATH_INFO: `/v3/projects/475af14af9fa44ee981907bde05963dc/users/289f38f1ad73467484f7d1710669ceab/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:36.660488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf4f0a-335d-4b5d-a16f-8676aa8bd730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=475af14af9fa44ee981907bde05963dc, user_id=289f38f1ad73467484f7d1710669ceab, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:36.663135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf4f0a-335d-4b5d-a16f-8676aa8bd730 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:36.673810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1308/2591] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:18:36 2022] PUT /identity/v3/projects/475af14af9fa44ee981907bde05963dc/users/289f38f1ad73467484f7d1710669ceab/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:37.399819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad4ad5-70cb-46f5-88d8-57bca302098c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:37.400331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad4ad5-70cb-46f5-88d8-57bca302098c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:37.400331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad4ad5-70cb-46f5-88d8-57bca302098c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:37.429272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4cad4ad5-70cb-46f5-88d8-57bca302098c None None] MFA Rules not processed for user `289f38f1ad73467484f7d1710669ceab`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:37.430501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4cad4ad5-70cb-46f5-88d8-57bca302098c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:37.508719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1284/2592] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:37 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:38.563382 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf4f0a-335d-4b5d-a16f-8676aa8bd730 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:38.634461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf4f0a-335d-4b5d-a16f-8676aa8bd730 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:38.639801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:38.640427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:38.640704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:38.640972 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] PATH_INFO: `/v3/users/8c2fd4d5ef9e4568851097bb4a6c01f9` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:38.642104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c2fd4d5ef9e4568851097bb4a6c01f9)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:38.644098 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:38.666926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:38.690977 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c2fd4d5ef9e4568851097bb4a6c01f9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:38.703394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c2fd4d5ef9e4568851097bb4a6c01f9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:38.710403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c2fd4d5ef9e4568851097bb4a6c01f9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:38.718492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c2fd4d5ef9e4568851097bb4a6c01f9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:38.719144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c2fd4d5ef9e4568851097bb4a6c01f9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:38.725228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1309/2593] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:18:38 2022] DELETE /identity/v3/users/8c2fd4d5ef9e4568851097bb4a6c01f9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:38.904294 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad4ad5-70cb-46f5-88d8-57bca302098c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:38.915974 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad4ad5-70cb-46f5-88d8-57bca302098c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:38.974313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:38.974859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:38.975193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:38.975457 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] PATH_INFO: `/v3/projects/8ba8c6057ca64042aed56a34c4b867df` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:38.976574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba8c6057ca64042aed56a34c4b867df)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:38.978470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:39.024331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba8c6057ca64042aed56a34c4b867df', 'request_id': 'req-19ef777c-cf18-44e1-8461-7457a2141cd3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.028994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:39.030337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba8c6057ca64042aed56a34c4b867df', 'request_id': 'req-19ef777c-cf18-44e1-8461-7457a2141cd3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.038053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba8c6057ca64042aed56a34c4b867df', 'request_id': 'req-19ef777c-cf18-44e1-8461-7457a2141cd3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.051000 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] The token cache is being invalidate because project 8ba8c6057ca64042aed56a34c4b867df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:18:39.051296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.051940 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.053422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1285/2594] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:18:38 2022] DELETE /identity/v3/projects/8ba8c6057ca64042aed56a34c4b867df => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:39.057566 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:39.059102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:39.072738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35eb8e-89b0-4cb5-a7fd-736decca913a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:39.129285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b992b-363c-4dbc-999b-856bfb2d0849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:39.129914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b992b-363c-4dbc-999b-856bfb2d0849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:39.130192 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b992b-363c-4dbc-999b-856bfb2d0849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:39.130465 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b992b-363c-4dbc-999b-856bfb2d0849 admin admin] PATH_INFO: `/v3/users/289f38f1ad73467484f7d1710669ceab` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:39.131613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b992b-363c-4dbc-999b-856bfb2d0849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=289f38f1ad73467484f7d1710669ceab)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:39.133560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b992b-363c-4dbc-999b-856bfb2d0849 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:39.179777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6a4b992b-363c-4dbc-999b-856bfb2d0849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '289f38f1ad73467484f7d1710669ceab'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.192217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6a4b992b-363c-4dbc-999b-856bfb2d0849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '289f38f1ad73467484f7d1710669ceab'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.199640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6a4b992b-363c-4dbc-999b-856bfb2d0849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '289f38f1ad73467484f7d1710669ceab'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.208829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6a4b992b-363c-4dbc-999b-856bfb2d0849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '289f38f1ad73467484f7d1710669ceab'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.209716 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6a4b992b-363c-4dbc-999b-856bfb2d0849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '289f38f1ad73467484f7d1710669ceab'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.219508 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1310/2595] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:18:39 2022] DELETE /identity/v3/users/289f38f1ad73467484f7d1710669ceab => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:39.394149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:39.395651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:39.408172 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef777c-cf18-44e1-8461-7457a2141cd3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:39.466254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:39.466886 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:39.467201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:39.467478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] PATH_INFO: `/v3/projects/475af14af9fa44ee981907bde05963dc` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:39.468589 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=475af14af9fa44ee981907bde05963dc)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:39.470493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:39.517534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '475af14af9fa44ee981907bde05963dc', 'request_id': 'req-8a7d4879-576b-4918-9792-2270f30cc4a1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.522880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '475af14af9fa44ee981907bde05963dc', 'request_id': 'req-8a7d4879-576b-4918-9792-2270f30cc4a1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.528957 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '475af14af9fa44ee981907bde05963dc', 'request_id': 'req-8a7d4879-576b-4918-9792-2270f30cc4a1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.542254 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] The token cache is being invalidate because project 475af14af9fa44ee981907bde05963dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:18:39.542548 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.543209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8a7d4879-576b-4918-9792-2270f30cc4a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:39.544815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1286/2596] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:18:39 2022] DELETE /identity/v3/projects/475af14af9fa44ee981907bde05963dc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:39.550476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef9d65-6b3e-4ae1-a370-58fc21f696ff None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:39.550763 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef9d65-6b3e-4ae1-a370-58fc21f696ff None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:39.551074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef9d65-6b3e-4ae1-a370-58fc21f696ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:39.571354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-71ef9d65-6b3e-4ae1-a370-58fc21f696ff None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:39.572546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-71ef9d65-6b3e-4ae1-a370-58fc21f696ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:39.585142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:39.635751 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1311/2597] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:39.643582 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154416df-4b46-4660-8e5d-2d116a35d1d2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:39.643872 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154416df-4b46-4660-8e5d-2d116a35d1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:39.644146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154416df-4b46-4660-8e5d-2d116a35d1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:39.665661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-154416df-4b46-4660-8e5d-2d116a35d1d2 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:39.666854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-154416df-4b46-4660-8e5d-2d116a35d1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:39.675609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1287/2598] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:39.679855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef9d65-6b3e-4ae1-a370-58fc21f696ff None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:39.689780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef9d65-6b3e-4ae1-a370-58fc21f696ff None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:39.694316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e4366-7493-4151-a212-9d3fa0dc527c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:39.694911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e4366-7493-4151-a212-9d3fa0dc527c admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:39.695365 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e4366-7493-4151-a212-9d3fa0dc527c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:39.695641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e4366-7493-4151-a212-9d3fa0dc527c admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:39.696309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e4366-7493-4151-a212-9d3fa0dc527c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:18:39.696611 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e4366-7493-4151-a212-9d3fa0dc527c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:39.698172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e4366-7493-4151-a212-9d3fa0dc527c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:39.710241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1312/2599] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:18:39.714535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154416df-4b46-4660-8e5d-2d116a35d1d2 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:39.724338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154416df-4b46-4660-8e5d-2d116a35d1d2 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:39.729021 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd45546-f744-47a6-a793-1800324c509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:39.729617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd45546-f744-47a6-a793-1800324c509b admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:39.729901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd45546-f744-47a6-a793-1800324c509b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:39.730176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd45546-f744-47a6-a793-1800324c509b admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:39.730803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd45546-f744-47a6-a793-1800324c509b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:39.732735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd45546-f744-47a6-a793-1800324c509b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:39.761050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1288/2600] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:18:39 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:18:39.765337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e4366-7493-4151-a212-9d3fa0dc527c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:39.776014 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e4366-7493-4151-a212-9d3fa0dc527c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:39.831682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cf59a-8bb7-4d59-a3b5-c2bb29505ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:39.832276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cf59a-8bb7-4d59-a3b5-c2bb29505ccf admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:39.832554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cf59a-8bb7-4d59-a3b5-c2bb29505ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:39.832823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cf59a-8bb7-4d59-a3b5-c2bb29505ccf admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:39.833479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cf59a-8bb7-4d59-a3b5-c2bb29505ccf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:39.835400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cf59a-8bb7-4d59-a3b5-c2bb29505ccf admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:39.857133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1313/2601] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:18:39 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:18:39.862078 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd45546-f744-47a6-a793-1800324c509b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:39.872520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd45546-f744-47a6-a793-1800324c509b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:39.877103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736260e6-98cb-439b-8179-1a3bdde6b96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:39.877698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736260e6-98cb-439b-8179-1a3bdde6b96f admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:39.877971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736260e6-98cb-439b-8179-1a3bdde6b96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:39.878259 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736260e6-98cb-439b-8179-1a3bdde6b96f admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:39.878869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736260e6-98cb-439b-8179-1a3bdde6b96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:39.880484 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736260e6-98cb-439b-8179-1a3bdde6b96f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:39.890635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1289/2602] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:39.894557 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cf59a-8bb7-4d59-a3b5-c2bb29505ccf admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:39.905485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cf59a-8bb7-4d59-a3b5-c2bb29505ccf admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:39.910148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad9c67-aefe-4378-b0b9-0c74b746bbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:39.910741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad9c67-aefe-4378-b0b9-0c74b746bbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:39.911061 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad9c67-aefe-4378-b0b9-0c74b746bbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:39.911365 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad9c67-aefe-4378-b0b9-0c74b746bbc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:39.912004 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad9c67-aefe-4378-b0b9-0c74b746bbc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:39.913571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad9c67-aefe-4378-b0b9-0c74b746bbc3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:39.925046 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1314/2603] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:39.929064 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736260e6-98cb-439b-8179-1a3bdde6b96f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:39.939162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736260e6-98cb-439b-8179-1a3bdde6b96f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:39.943698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a29b45-0a91-4621-8a51-080a8f0dfcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:39.944333 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a29b45-0a91-4621-8a51-080a8f0dfcf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:39.944610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a29b45-0a91-4621-8a51-080a8f0dfcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:39.944881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a29b45-0a91-4621-8a51-080a8f0dfcf7 admin admin] PATH_INFO: `/v3/projects/cbf41d53dfab4fa8856fa5600695f026/users/babd0747d1d84098b319bbeb1c06fefa/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:39.959918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a29b45-0a91-4621-8a51-080a8f0dfcf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf41d53dfab4fa8856fa5600695f026, user_id=babd0747d1d84098b319bbeb1c06fefa, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:39.962731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a29b45-0a91-4621-8a51-080a8f0dfcf7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:39.976213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1290/2604] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:18:39 2022] PUT /identity/v3/projects/cbf41d53dfab4fa8856fa5600695f026/users/babd0747d1d84098b319bbeb1c06fefa/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:18:39.987345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70217f45-04cb-4167-bd7b-72d8517c2b1b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:39.987636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70217f45-04cb-4167-bd7b-72d8517c2b1b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:39.987936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70217f45-04cb-4167-bd7b-72d8517c2b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:40.015645 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-70217f45-04cb-4167-bd7b-72d8517c2b1b None None] MFA Rules not processed for user `babd0747d1d84098b319bbeb1c06fefa`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:40.016853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-70217f45-04cb-4167-bd7b-72d8517c2b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:40.081831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1315/2605] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:40.086237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a29b45-0a91-4621-8a51-080a8f0dfcf7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:40.100187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:40.150896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a29b45-0a91-4621-8a51-080a8f0dfcf7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:40.156205 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e31644-c815-42ed-843b-0c39c655a704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:40.156809 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e31644-c815-42ed-843b-0c39c655a704 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:40.157095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e31644-c815-42ed-843b-0c39c655a704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:40.157366 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e31644-c815-42ed-843b-0c39c655a704 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:40.157988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e31644-c815-42ed-843b-0c39c655a704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:40.159930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e31644-c815-42ed-843b-0c39c655a704 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:40.185786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1291/2606] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:18:40 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:18:40.189926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70217f45-04cb-4167-bd7b-72d8517c2b1b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:40.200923 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70217f45-04cb-4167-bd7b-72d8517c2b1b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:40.213142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:40.264806 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf68341-8c75-4241-8f2f-8545eba1a514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:40.265604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf68341-8c75-4241-8f2f-8545eba1a514 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:40.265863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf68341-8c75-4241-8f2f-8545eba1a514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:40.266254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf68341-8c75-4241-8f2f-8545eba1a514 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:40.267037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf68341-8c75-4241-8f2f-8545eba1a514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:40.269477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf68341-8c75-4241-8f2f-8545eba1a514 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:40.290999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1316/2607] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:18:40 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:18:40.295049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e31644-c815-42ed-843b-0c39c655a704 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:40.306223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e31644-c815-42ed-843b-0c39c655a704 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:40.311539 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c8e66-271d-44d5-8454-3c48bac77698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:40.312146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c8e66-271d-44d5-8454-3c48bac77698 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:40.312430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c8e66-271d-44d5-8454-3c48bac77698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:40.312700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c8e66-271d-44d5-8454-3c48bac77698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:40.313343 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c8e66-271d-44d5-8454-3c48bac77698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:40.315030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c8e66-271d-44d5-8454-3c48bac77698 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:40.328238 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1292/2608] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:40.332516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf68341-8c75-4241-8f2f-8545eba1a514 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:40.342749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf68341-8c75-4241-8f2f-8545eba1a514 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:40.347664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b6dc6-e6de-4dd6-8020-41f534007d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:40.348316 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b6dc6-e6de-4dd6-8020-41f534007d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:40.348597 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b6dc6-e6de-4dd6-8020-41f534007d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:40.348873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b6dc6-e6de-4dd6-8020-41f534007d77 admin admin] PATH_INFO: `/v3/projects/b9531e5821014fc8b481b1554d7bf020/users/90702196be4d4f729ccedd8e69d53166/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:40.364347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b6dc6-e6de-4dd6-8020-41f534007d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9531e5821014fc8b481b1554d7bf020, user_id=90702196be4d4f729ccedd8e69d53166, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:40.366971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b6dc6-e6de-4dd6-8020-41f534007d77 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:40.380786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1317/2609] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:18:40 2022] PUT /identity/v3/projects/b9531e5821014fc8b481b1554d7bf020/users/90702196be4d4f729ccedd8e69d53166/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:40.384706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c8e66-271d-44d5-8454-3c48bac77698 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:40.395388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c8e66-271d-44d5-8454-3c48bac77698 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:40.452285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352c99c-423b-4023-87e3-84ea6ff77050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:40.452894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352c99c-423b-4023-87e3-84ea6ff77050 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:40.453175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352c99c-423b-4023-87e3-84ea6ff77050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:40.453450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352c99c-423b-4023-87e3-84ea6ff77050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:40.454082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352c99c-423b-4023-87e3-84ea6ff77050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:40.455713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352c99c-423b-4023-87e3-84ea6ff77050 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:40.465317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1293/2610] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:40.469625 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b6dc6-e6de-4dd6-8020-41f534007d77 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:40.480394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b6dc6-e6de-4dd6-8020-41f534007d77 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:40.486533 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaba669-91c6-45bc-8db6-6fb7328ee14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:40.487703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba669-91c6-45bc-8db6-6fb7328ee14a admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:40.488207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba669-91c6-45bc-8db6-6fb7328ee14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:40.488708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba669-91c6-45bc-8db6-6fb7328ee14a admin admin] PATH_INFO: `/v3/projects/b9531e5821014fc8b481b1554d7bf020/users/90702196be4d4f729ccedd8e69d53166/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:40.490700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaba669-91c6-45bc-8db6-6fb7328ee14a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9531e5821014fc8b481b1554d7bf020, user_id=90702196be4d4f729ccedd8e69d53166, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:40.493513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaba669-91c6-45bc-8db6-6fb7328ee14a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:40.507856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1318/2611] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:18:40 2022] PUT /identity/v3/projects/b9531e5821014fc8b481b1554d7bf020/users/90702196be4d4f729ccedd8e69d53166/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:18:40.523060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598240f-4d49-4a3f-a14c-2036ef426234 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:40.523342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598240f-4d49-4a3f-a14c-2036ef426234 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:40.523611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598240f-4d49-4a3f-a14c-2036ef426234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:40.554881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5598240f-4d49-4a3f-a14c-2036ef426234 None None] MFA Rules not processed for user `90702196be4d4f729ccedd8e69d53166`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:40.556119 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5598240f-4d49-4a3f-a14c-2036ef426234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:40.623210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1294/2612] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:40.634984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaba669-91c6-45bc-8db6-6fb7328ee14a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:40.636734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-acaba669-91c6-45bc-8db6-6fb7328ee14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:40.695788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaba669-91c6-45bc-8db6-6fb7328ee14a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:40.699406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf873c7-171c-4fa7-bd98-dce6f85e5db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:40.700001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf873c7-171c-4fa7-bd98-dce6f85e5db0 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:40.700271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf873c7-171c-4fa7-bd98-dce6f85e5db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:40.700545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf873c7-171c-4fa7-bd98-dce6f85e5db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:40.709444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf873c7-171c-4fa7-bd98-dce6f85e5db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:40.710835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf873c7-171c-4fa7-bd98-dce6f85e5db0 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:40.720367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1319/2613] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:40.796491 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598240f-4d49-4a3f-a14c-2036ef426234 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:40.806927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598240f-4d49-4a3f-a14c-2036ef426234 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:40.810568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62858f4c-393c-44de-998c-0dfd7c899b6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:40.811860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62858f4c-393c-44de-998c-0dfd7c899b6c service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:40.813062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62858f4c-393c-44de-998c-0dfd7c899b6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:40.813062 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62858f4c-393c-44de-998c-0dfd7c899b6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:40.816976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:40.870689 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62858f4c-393c-44de-998c-0dfd7c899b6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:40.872298 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62858f4c-393c-44de-998c-0dfd7c899b6c service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:40.882218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1295/2614] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:50.716025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=63,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:50.878081 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=25 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:57.891505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf873c7-171c-4fa7-bd98-dce6f85e5db0 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:57.893201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bcf873c7-171c-4fa7-bd98-dce6f85e5db0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:57.971657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf873c7-171c-4fa7-bd98-dce6f85e5db0 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:57.976152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:57.976758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:57.977032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:57.977301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] PATH_INFO: `/v3/users/d0ecac93f4784e37bbb1858c2a16dcfa` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:57.978418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0ecac93f4784e37bbb1858c2a16dcfa)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:57.980387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:58.028436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0ecac93f4784e37bbb1858c2a16dcfa'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.040719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0ecac93f4784e37bbb1858c2a16dcfa'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.048149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0ecac93f4784e37bbb1858c2a16dcfa'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.057694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0ecac93f4784e37bbb1858c2a16dcfa'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.058632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0ecac93f4784e37bbb1858c2a16dcfa'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.066219 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1320/2615] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:18:57 2022] DELETE /identity/v3/users/d0ecac93f4784e37bbb1858c2a16dcfa => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:18:58.070706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae8657-fd5a-40fd-bb30-13738e9d759b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:58.071221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae8657-fd5a-40fd-bb30-13738e9d759b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:58.071321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae8657-fd5a-40fd-bb30-13738e9d759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:58.090580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d7ae8657-fd5a-40fd-bb30-13738e9d759b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:58.091818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d7ae8657-fd5a-40fd-bb30-13738e9d759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:58.149329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1296/2616] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:58.159633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:58.161030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:58.222433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ec989-ff48-4540-81e3-c992c0aed42a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:58.226400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c40bd0-1910-4ed3-9113-28dae28d0415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:58.227101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c40bd0-1910-4ed3-9113-28dae28d0415 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:58.227376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c40bd0-1910-4ed3-9113-28dae28d0415 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:58.227648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c40bd0-1910-4ed3-9113-28dae28d0415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:58.238624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c40bd0-1910-4ed3-9113-28dae28d0415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:58.240156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c40bd0-1910-4ed3-9113-28dae28d0415 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:58.249622 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1321/2617] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:58.521944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae8657-fd5a-40fd-bb30-13738e9d759b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:58.523466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d7ae8657-fd5a-40fd-bb30-13738e9d759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:58.536911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae8657-fd5a-40fd-bb30-13738e9d759b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:58.541845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:58.542486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:58.542769 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:58.543079 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] PATH_INFO: `/v3/projects/be8d41d150dd49538dee8c656928e05b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:58.544416 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be8d41d150dd49538dee8c656928e05b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:58.546233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:58.599771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be8d41d150dd49538dee8c656928e05b', 'request_id': 'req-3a0a1d97-d53c-46ff-b52f-4b82faa24587'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.605025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be8d41d150dd49538dee8c656928e05b', 'request_id': 'req-3a0a1d97-d53c-46ff-b52f-4b82faa24587'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.611495 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be8d41d150dd49538dee8c656928e05b', 'request_id': 'req-3a0a1d97-d53c-46ff-b52f-4b82faa24587'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.626465 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] The token cache is being invalidate because project be8d41d150dd49538dee8c656928e05b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:18:58.626762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.627486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.629219 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1297/2618] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:18:58 2022] DELETE /identity/v3/projects/be8d41d150dd49538dee8c656928e05b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:58.633172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c40bd0-1910-4ed3-9113-28dae28d0415 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:58.643733 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c40bd0-1910-4ed3-9113-28dae28d0415 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:58.664970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:58.708879 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cfd42-f359-4c9b-82ee-082edb3d83a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:58.709515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cfd42-f359-4c9b-82ee-082edb3d83a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:58.709796 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cfd42-f359-4c9b-82ee-082edb3d83a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:58.710068 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cfd42-f359-4c9b-82ee-082edb3d83a6 admin admin] PATH_INFO: `/v3/users/e0e98b552b3e4208b5bfdc023ef7fcd6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:58.711684 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cfd42-f359-4c9b-82ee-082edb3d83a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e98b552b3e4208b5bfdc023ef7fcd6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:58.713614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cfd42-f359-4c9b-82ee-082edb3d83a6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:58.762186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:58.763617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:58.769513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1f5cfd42-f359-4c9b-82ee-082edb3d83a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e98b552b3e4208b5bfdc023ef7fcd6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.778260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a1d97-d53c-46ff-b52f-4b82faa24587 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:58.781558 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1f5cfd42-f359-4c9b-82ee-082edb3d83a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e98b552b3e4208b5bfdc023ef7fcd6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.783262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:58.783899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:58.784279 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:58.784659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] PATH_INFO: `/v3/users/babd0747d1d84098b319bbeb1c06fefa` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:58.785848 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] RBAC: Authorizing `identity:delete_user(user_id=babd0747d1d84098b319bbeb1c06fefa)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:58.787882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:58.788790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1f5cfd42-f359-4c9b-82ee-082edb3d83a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e98b552b3e4208b5bfdc023ef7fcd6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.796454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1f5cfd42-f359-4c9b-82ee-082edb3d83a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e98b552b3e4208b5bfdc023ef7fcd6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.797227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1f5cfd42-f359-4c9b-82ee-082edb3d83a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e98b552b3e4208b5bfdc023ef7fcd6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.804008 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1322/2619] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:18:58 2022] DELETE /identity/v3/users/e0e98b552b3e4208b5bfdc023ef7fcd6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:58.838579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'babd0747d1d84098b319bbeb1c06fefa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.855317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'babd0747d1d84098b319bbeb1c06fefa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.862989 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'babd0747d1d84098b319bbeb1c06fefa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.863882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'babd0747d1d84098b319bbeb1c06fefa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.868361 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:58.873388 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'babd0747d1d84098b319bbeb1c06fefa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:58.884798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1298/2620] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:18:58 2022] DELETE /identity/v3/users/babd0747d1d84098b319bbeb1c06fefa => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:18:58.890110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0af09-20c5-4608-a49f-38a4b708d090 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:58.891231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0af09-20c5-4608-a49f-38a4b708d090 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:58.891694 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0af09-20c5-4608-a49f-38a4b708d090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:58.912133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8ac0af09-20c5-4608-a49f-38a4b708d090 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:58.913354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8ac0af09-20c5-4608-a49f-38a4b708d090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:58.974655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1323/2621] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:58.984391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:58.986037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:59.048586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49be2a-6b7e-4a27-ac60-9ff71b1bcdde admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.052675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050aeb55-9f34-412c-8fa0-22707ec46d90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:59.053320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050aeb55-9f34-412c-8fa0-22707ec46d90 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.053629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050aeb55-9f34-412c-8fa0-22707ec46d90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.053906 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050aeb55-9f34-412c-8fa0-22707ec46d90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.063790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050aeb55-9f34-412c-8fa0-22707ec46d90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:59.065201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050aeb55-9f34-412c-8fa0-22707ec46d90 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:59.076429 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1299/2622] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:18:59.089265 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0af09-20c5-4608-a49f-38a4b708d090 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.091218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8ac0af09-20c5-4608-a49f-38a4b708d090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:59.110748 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0af09-20c5-4608-a49f-38a4b708d090 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.116740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:59.117732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.118020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.118295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] PATH_INFO: `/v3/projects/e70d7cc2bc9f4b9d94143f9828138fad` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.119856 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70d7cc2bc9f4b9d94143f9828138fad)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:59.122223 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:59.165763 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:59.177584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70d7cc2bc9f4b9d94143f9828138fad', 'request_id': 'req-1616e204-f9da-4c4a-a539-ad135851ce3a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.186541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70d7cc2bc9f4b9d94143f9828138fad', 'request_id': 'req-1616e204-f9da-4c4a-a539-ad135851ce3a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.193200 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70d7cc2bc9f4b9d94143f9828138fad', 'request_id': 'req-1616e204-f9da-4c4a-a539-ad135851ce3a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.206941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] The token cache is being invalidate because project e70d7cc2bc9f4b9d94143f9828138fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:18:59.207282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.207908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1616e204-f9da-4c4a-a539-ad135851ce3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.209413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1324/2623] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:18:59 2022] DELETE /identity/v3/projects/e70d7cc2bc9f4b9d94143f9828138fad => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:59.215210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084eadb3-1cc8-49e3-b84a-0e167137a152 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.215496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084eadb3-1cc8-49e3-b84a-0e167137a152 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.215771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084eadb3-1cc8-49e3-b84a-0e167137a152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.236463 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-084eadb3-1cc8-49e3-b84a-0e167137a152 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:59.237688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-084eadb3-1cc8-49e3-b84a-0e167137a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:59.304184 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1300/2624] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:59.313281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463f53b-71e6-4d50-acaa-756477ee2844 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.313565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463f53b-71e6-4d50-acaa-756477ee2844 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.313836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463f53b-71e6-4d50-acaa-756477ee2844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.334302 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c463f53b-71e6-4d50-acaa-756477ee2844 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:59.335794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c463f53b-71e6-4d50-acaa-756477ee2844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:59.345105 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1325/2625] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:59.349967 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084eadb3-1cc8-49e3-b84a-0e167137a152 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.358705 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463f53b-71e6-4d50-acaa-756477ee2844 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.360745 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c463f53b-71e6-4d50-acaa-756477ee2844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:59.361420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084eadb3-1cc8-49e3-b84a-0e167137a152 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.370054 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e6f07-ce44-41c9-9550-cc3c15493b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:59.370672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e6f07-ce44-41c9-9550-cc3c15493b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.370965 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e6f07-ce44-41c9-9550-cc3c15493b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.371245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e6f07-ce44-41c9-9550-cc3c15493b88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.372180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e6f07-ce44-41c9-9550-cc3c15493b88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:18:59.372180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e6f07-ce44-41c9-9550-cc3c15493b88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:59.372892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463f53b-71e6-4d50-acaa-756477ee2844 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.373802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e6f07-ce44-41c9-9550-cc3c15493b88 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:59.378250 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:59.378923 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.379245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.379497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] PATH_INFO: `/v3/projects/cbf41d53dfab4fa8856fa5600695f026` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.380762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf41d53dfab4fa8856fa5600695f026)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:59.382776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:59.389703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1301/2626] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 17:18:59.390972 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e6f07-ce44-41c9-9550-cc3c15493b88 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.402142 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e6f07-ce44-41c9-9550-cc3c15493b88 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.408609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e19ce-9307-4d41-995d-cb3441fbfeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:59.409232 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e19ce-9307-4d41-995d-cb3441fbfeb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.409651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e19ce-9307-4d41-995d-cb3441fbfeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.409928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e19ce-9307-4d41-995d-cb3441fbfeb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.410560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e19ce-9307-4d41-995d-cb3441fbfeb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:59.412687 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e19ce-9307-4d41-995d-cb3441fbfeb8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:59.435556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf41d53dfab4fa8856fa5600695f026', 'request_id': 'req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.441852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1302/2627] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:18:59 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:18:59.445858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf41d53dfab4fa8856fa5600695f026', 'request_id': 'req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.446020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e19ce-9307-4d41-995d-cb3441fbfeb8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.452503 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf41d53dfab4fa8856fa5600695f026', 'request_id': 'req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.467240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] The token cache is being invalidate because project cbf41d53dfab4fa8856fa5600695f026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:18:59.467528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.468158 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.469679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1326/2628] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:18:59 2022] DELETE /identity/v3/projects/cbf41d53dfab4fa8856fa5600695f026 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:18:59.473555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.475284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:59.516033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e19ce-9307-4d41-995d-cb3441fbfeb8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.520732 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d02673-29ca-4853-97bd-7b26e5ad1116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:59.521330 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d02673-29ca-4853-97bd-7b26e5ad1116 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.521609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d02673-29ca-4853-97bd-7b26e5ad1116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.521907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d02673-29ca-4853-97bd-7b26e5ad1116 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.522532 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d02673-29ca-4853-97bd-7b26e5ad1116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:59.524608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d02673-29ca-4853-97bd-7b26e5ad1116 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:59.536067 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:59.549154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37a0de-9d6d-4d44-81e3-aceb1701ab9f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.549935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1303/2629] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:18:59 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:18:59.553572 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d02673-29ca-4853-97bd-7b26e5ad1116 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.554585 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cc9e8-cc26-4f18-b0f6-8267aa7287bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:59.555212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-97d02673-29ca-4853-97bd-7b26e5ad1116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:59.555355 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cc9e8-cc26-4f18-b0f6-8267aa7287bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.555531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cc9e8-cc26-4f18-b0f6-8267aa7287bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.555802 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cc9e8-cc26-4f18-b0f6-8267aa7287bf admin admin] PATH_INFO: `/v3/users/90702196be4d4f729ccedd8e69d53166` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.556959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cc9e8-cc26-4f18-b0f6-8267aa7287bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=90702196be4d4f729ccedd8e69d53166)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:59.558876 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cc9e8-cc26-4f18-b0f6-8267aa7287bf admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:59.567131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d02673-29ca-4853-97bd-7b26e5ad1116 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.572053 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee54eb-2b12-4340-ac72-22b5f71c7726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:59.572659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee54eb-2b12-4340-ac72-22b5f71c7726 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.572940 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee54eb-2b12-4340-ac72-22b5f71c7726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.573210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee54eb-2b12-4340-ac72-22b5f71c7726 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.573847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee54eb-2b12-4340-ac72-22b5f71c7726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:59.575521 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee54eb-2b12-4340-ac72-22b5f71c7726 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:59.586263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1304/2630] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:18:59.593350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee54eb-2b12-4340-ac72-22b5f71c7726 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.604065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee54eb-2b12-4340-ac72-22b5f71c7726 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.610489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d408ad-da2a-43ab-b788-e72b337e7006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:59.611143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d408ad-da2a-43ab-b788-e72b337e7006 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.611573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d408ad-da2a-43ab-b788-e72b337e7006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.611786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d408ad-da2a-43ab-b788-e72b337e7006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.612425 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d408ad-da2a-43ab-b788-e72b337e7006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:59.614100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d408ad-da2a-43ab-b788-e72b337e7006 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:59.623497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:18:59.626371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1305/2631] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:18:59.627407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-165cc9e8-cc26-4f18-b0f6-8267aa7287bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90702196be4d4f729ccedd8e69d53166'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.630451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d408ad-da2a-43ab-b788-e72b337e7006 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.640976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d408ad-da2a-43ab-b788-e72b337e7006 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.643507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-165cc9e8-cc26-4f18-b0f6-8267aa7287bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90702196be4d4f729ccedd8e69d53166'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.647358 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0be3ec-4bd0-4ff3-917e-55cfa705cf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:59.648096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0be3ec-4bd0-4ff3-917e-55cfa705cf4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.648373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0be3ec-4bd0-4ff3-917e-55cfa705cf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.648641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0be3ec-4bd0-4ff3-917e-55cfa705cf4a admin admin] PATH_INFO: `/v3/projects/a6daee4eb75742c2a917560bae62cede/users/2ed8f99b53104681af7d148f88913e2f/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.651171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-165cc9e8-cc26-4f18-b0f6-8267aa7287bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90702196be4d4f729ccedd8e69d53166'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.663807 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-165cc9e8-cc26-4f18-b0f6-8267aa7287bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90702196be4d4f729ccedd8e69d53166'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.665721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-165cc9e8-cc26-4f18-b0f6-8267aa7287bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90702196be4d4f729ccedd8e69d53166'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:18:59.669828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0be3ec-4bd0-4ff3-917e-55cfa705cf4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6daee4eb75742c2a917560bae62cede, user_id=2ed8f99b53104681af7d148f88913e2f, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:59.674015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1327/2632] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:18:59 2022] DELETE /identity/v3/users/90702196be4d4f729ccedd8e69d53166 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:18:59.674199 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0be3ec-4bd0-4ff3-917e-55cfa705cf4a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:59.691997 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1306/2633] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:18:59 2022] PUT /identity/v3/projects/a6daee4eb75742c2a917560bae62cede/users/2ed8f99b53104681af7d148f88913e2f/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:18:59.703413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614988e-c428-49d4-b5c6-b3d791abc399 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.703783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614988e-c428-49d4-b5c6-b3d791abc399 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.704149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614988e-c428-49d4-b5c6-b3d791abc399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.735653 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6614988e-c428-49d4-b5c6-b3d791abc399 None None] MFA Rules not processed for user `2ed8f99b53104681af7d148f88913e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:18:59.737214 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6614988e-c428-49d4-b5c6-b3d791abc399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:59.813650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1328/2634] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:18:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:18:59.818626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0be3ec-4bd0-4ff3-917e-55cfa705cf4a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.820371 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-eb0be3ec-4bd0-4ff3-917e-55cfa705cf4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:18:59.898584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0be3ec-4bd0-4ff3-917e-55cfa705cf4a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.903505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b5e1e-478d-463e-b989-29806d8b7e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:18:59.904167 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5e1e-478d-463e-b989-29806d8b7e41 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:18:59.904447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5e1e-478d-463e-b989-29806d8b7e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:18:59.904714 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5e1e-478d-463e-b989-29806d8b7e41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:18:59.905376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b5e1e-478d-463e-b989-29806d8b7e41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:18:59.907934 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b5e1e-478d-463e-b989-29806d8b7e41 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:18:59.937687 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1307/2635] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:18:59 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:18:59.939282 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614988e-c428-49d4-b5c6-b3d791abc399 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.957258 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614988e-c428-49d4-b5c6-b3d791abc399 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:18:59.963979 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b5e1e-478d-463e-b989-29806d8b7e41 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:18:59.966137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2b1b5e1e-478d-463e-b989-29806d8b7e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:00.026053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e238f-8621-454a-9b32-03a8d11e9da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.026669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e238f-8621-454a-9b32-03a8d11e9da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.026974 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e238f-8621-454a-9b32-03a8d11e9da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.027270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e238f-8621-454a-9b32-03a8d11e9da4 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.027917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e238f-8621-454a-9b32-03a8d11e9da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.029888 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e238f-8621-454a-9b32-03a8d11e9da4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.040237 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b5e1e-478d-463e-b989-29806d8b7e41 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.045255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.045891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.046339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.046463 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] PATH_INFO: `/v3/projects/b9531e5821014fc8b481b1554d7bf020` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.047798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9531e5821014fc8b481b1554d7bf020)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.049965 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.052319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1329/2636] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:18:59 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:19:00.056239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e238f-8621-454a-9b32-03a8d11e9da4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.065874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e238f-8621-454a-9b32-03a8d11e9da4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.070831 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22873-c639-479c-aeca-b8c919320d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.071453 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22873-c639-479c-aeca-b8c919320d32 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.071729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22873-c639-479c-aeca-b8c919320d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.072003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22873-c639-479c-aeca-b8c919320d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.073082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf22873-c639-479c-aeca-b8c919320d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.074674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf22873-c639-479c-aeca-b8c919320d32 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.085941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1330/2637] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:00.090164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22873-c639-479c-aeca-b8c919320d32 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.090978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:INSERT=3,keystone:SELECT=95,keystone:DELETE=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:00.100931 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22873-c639-479c-aeca-b8c919320d32 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.103571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9531e5821014fc8b481b1554d7bf020', 'request_id': 'req-b4ac05a4-11d9-40f7-806b-8444f1f4be51'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:00.109534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9531e5821014fc8b481b1554d7bf020', 'request_id': 'req-b4ac05a4-11d9-40f7-806b-8444f1f4be51'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:00.116006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9531e5821014fc8b481b1554d7bf020', 'request_id': 'req-b4ac05a4-11d9-40f7-806b-8444f1f4be51'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:00.127544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:00.129883 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] The token cache is being invalidate because project b9531e5821014fc8b481b1554d7bf020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:00.130308 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:00.130995 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b4ac05a4-11d9-40f7-806b-8444f1f4be51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:00.132486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1308/2638] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:18:59 2022] DELETE /identity/v3/projects/b9531e5821014fc8b481b1554d7bf020 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:00.137393 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed4597-6f5d-4ea2-9c07-1e463ce374b1 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.137726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed4597-6f5d-4ea2-9c07-1e463ce374b1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.138027 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed4597-6f5d-4ea2-9c07-1e463ce374b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.158229 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d2ed4597-6f5d-4ea2-9c07-1e463ce374b1 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:00.159468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d2ed4597-6f5d-4ea2-9c07-1e463ce374b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:00.165301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4ebfe-aa49-482e-a5b9-ebe6ae49465e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.165959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4ebfe-aa49-482e-a5b9-ebe6ae49465e admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.166241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4ebfe-aa49-482e-a5b9-ebe6ae49465e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.166515 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4ebfe-aa49-482e-a5b9-ebe6ae49465e admin admin] PATH_INFO: `/v3/projects/925f0f6024de42518e0f00d3558210d9/users/5836d2ac465f4032a6af48ca68d7ae7f/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.181783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4ebfe-aa49-482e-a5b9-ebe6ae49465e admin admin] RBAC: Authorizing `identity:create_grant(project_id=925f0f6024de42518e0f00d3558210d9, user_id=5836d2ac465f4032a6af48ca68d7ae7f, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.184438 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4ebfe-aa49-482e-a5b9-ebe6ae49465e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.198342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1331/2639] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:00 2022] PUT /identity/v3/projects/925f0f6024de42518e0f00d3558210d9/users/5836d2ac465f4032a6af48ca68d7ae7f/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:00.202162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4ebfe-aa49-482e-a5b9-ebe6ae49465e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.203884 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9cb4ebfe-aa49-482e-a5b9-ebe6ae49465e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:00.225481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1309/2640] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:19:00.233704 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2529c3c-0750-4825-983c-a91cf4788ab4 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.234560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2529c3c-0750-4825-983c-a91cf4788ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.234841 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2529c3c-0750-4825-983c-a91cf4788ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.254686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c2529c3c-0750-4825-983c-a91cf4788ab4 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:00.255922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c2529c3c-0750-4825-983c-a91cf4788ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:00.269109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4ebfe-aa49-482e-a5b9-ebe6ae49465e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.273940 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da09cf-e74b-4570-9a0d-cf6f5cceb491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.274545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da09cf-e74b-4570-9a0d-cf6f5cceb491 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.274823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da09cf-e74b-4570-9a0d-cf6f5cceb491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.275303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da09cf-e74b-4570-9a0d-cf6f5cceb491 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.275955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da09cf-e74b-4570-9a0d-cf6f5cceb491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.277690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da09cf-e74b-4570-9a0d-cf6f5cceb491 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.288945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1332/2641] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:00.293175 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da09cf-e74b-4570-9a0d-cf6f5cceb491 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.304879 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da09cf-e74b-4570-9a0d-cf6f5cceb491 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.309715 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbfd12-f9e9-427e-95a4-99f79510f3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.310366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbfd12-f9e9-427e-95a4-99f79510f3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.310654 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbfd12-f9e9-427e-95a4-99f79510f3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.310959 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbfd12-f9e9-427e-95a4-99f79510f3d8 admin admin] PATH_INFO: `/v3/projects/925f0f6024de42518e0f00d3558210d9/users/5836d2ac465f4032a6af48ca68d7ae7f/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.312938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbfd12-f9e9-427e-95a4-99f79510f3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=925f0f6024de42518e0f00d3558210d9, user_id=5836d2ac465f4032a6af48ca68d7ae7f, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.315602 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbfd12-f9e9-427e-95a4-99f79510f3d8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.318825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1310/2642] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:19:00.323123 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2529c3c-0750-4825-983c-a91cf4788ab4 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.329543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1333/2643] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:00 2022] PUT /identity/v3/projects/925f0f6024de42518e0f00d3558210d9/users/5836d2ac465f4032a6af48ca68d7ae7f/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:00.333086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2529c3c-0750-4825-983c-a91cf4788ab4 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.339938 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa0566-0598-48dd-883c-2d42a40be489 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.340229 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa0566-0598-48dd-883c-2d42a40be489 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.340504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa0566-0598-48dd-883c-2d42a40be489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.369475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b3aa0566-0598-48dd-883c-2d42a40be489 None None] MFA Rules not processed for user `5836d2ac465f4032a6af48ca68d7ae7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:00.370693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b3aa0566-0598-48dd-883c-2d42a40be489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:00.390339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c289a1-45e1-4077-b740-b86357d62f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.390825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c289a1-45e1-4077-b740-b86357d62f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.391521 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c289a1-45e1-4077-b740-b86357d62f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.391808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c289a1-45e1-4077-b740-b86357d62f17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.392486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c289a1-45e1-4077-b740-b86357d62f17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:19:00.392822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c289a1-45e1-4077-b740-b86357d62f17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.394654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c289a1-45e1-4077-b740-b86357d62f17 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.401849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:00.411140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1311/2644] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:19:00.412987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c289a1-45e1-4077-b740-b86357d62f17 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.423522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c289a1-45e1-4077-b740-b86357d62f17 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.428526 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e95d89-0947-4436-9f49-02cc4593611e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.429355 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e95d89-0947-4436-9f49-02cc4593611e admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.429754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e95d89-0947-4436-9f49-02cc4593611e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.430036 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e95d89-0947-4436-9f49-02cc4593611e admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.430695 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e95d89-0947-4436-9f49-02cc4593611e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.432697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e95d89-0947-4436-9f49-02cc4593611e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.443329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1334/2645] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:19:00.455373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa0566-0598-48dd-883c-2d42a40be489 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.457108 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b3aa0566-0598-48dd-883c-2d42a40be489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:00.461273 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1312/2646] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:00 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:19:00.465338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e95d89-0947-4436-9f49-02cc4593611e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.479774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:00.520055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa0566-0598-48dd-883c-2d42a40be489 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.530096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e95d89-0947-4436-9f49-02cc4593611e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.536037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40511917-aafe-4e43-9007-b20e3baffa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.536762 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40511917-aafe-4e43-9007-b20e3baffa7b admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.537041 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40511917-aafe-4e43-9007-b20e3baffa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.537311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40511917-aafe-4e43-9007-b20e3baffa7b admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.537947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40511917-aafe-4e43-9007-b20e3baffa7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.539894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40511917-aafe-4e43-9007-b20e3baffa7b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.565455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1313/2647] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:00 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:19:00.566639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40511917-aafe-4e43-9007-b20e3baffa7b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.572358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d9d29-526f-4294-ae89-0e4a26272606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.572986 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d9d29-526f-4294-ae89-0e4a26272606 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.573271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d9d29-526f-4294-ae89-0e4a26272606 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.573571 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d9d29-526f-4294-ae89-0e4a26272606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.576503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40511917-aafe-4e43-9007-b20e3baffa7b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.582028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372ee28-7158-4345-92c2-37ba667a11f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.582613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372ee28-7158-4345-92c2-37ba667a11f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.582892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372ee28-7158-4345-92c2-37ba667a11f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.583195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372ee28-7158-4345-92c2-37ba667a11f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.583837 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372ee28-7158-4345-92c2-37ba667a11f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.585422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372ee28-7158-4345-92c2-37ba667a11f4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.595934 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1314/2648] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:00.599910 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372ee28-7158-4345-92c2-37ba667a11f4 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.610124 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372ee28-7158-4345-92c2-37ba667a11f4 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.615066 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e941f4-b853-4e95-9cd6-f4d9bab482c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.615680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e941f4-b853-4e95-9cd6-f4d9bab482c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.615954 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e941f4-b853-4e95-9cd6-f4d9bab482c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.616218 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e941f4-b853-4e95-9cd6-f4d9bab482c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.616849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e941f4-b853-4e95-9cd6-f4d9bab482c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.618586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e941f4-b853-4e95-9cd6-f4d9bab482c7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.629754 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1315/2649] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:00.633951 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e941f4-b853-4e95-9cd6-f4d9bab482c7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.640899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d9d29-526f-4294-ae89-0e4a26272606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.642299 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d9d29-526f-4294-ae89-0e4a26272606 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.643944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e941f4-b853-4e95-9cd6-f4d9bab482c7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.648628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1946e-cf64-42c6-9f36-ee6d5676fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.649271 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1946e-cf64-42c6-9f36-ee6d5676fd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.649550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1946e-cf64-42c6-9f36-ee6d5676fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.649818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1946e-cf64-42c6-9f36-ee6d5676fd0e admin admin] PATH_INFO: `/v3/projects/60b72c76d808413894f0fbd90fbaf499/users/a971d893d9934e72b271a1a44d040ad4/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.651873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1335/2650] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:00.665257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1946e-cf64-42c6-9f36-ee6d5676fd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b72c76d808413894f0fbd90fbaf499, user_id=a971d893d9934e72b271a1a44d040ad4, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.670765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1946e-cf64-42c6-9f36-ee6d5676fd0e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.685175 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1316/2651] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:00 2022] PUT /identity/v3/projects/60b72c76d808413894f0fbd90fbaf499/users/a971d893d9934e72b271a1a44d040ad4/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:00.696808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f957585-fba4-4a27-9bc3-76341863fedd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.697171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f957585-fba4-4a27-9bc3-76341863fedd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.697528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f957585-fba4-4a27-9bc3-76341863fedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.760987 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1946e-cf64-42c6-9f36-ee6d5676fd0e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.821961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1946e-cf64-42c6-9f36-ee6d5676fd0e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.825577 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba2d9d-8b02-4400-ae3b-c875100fcb68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.826183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba2d9d-8b02-4400-ae3b-c875100fcb68 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.826459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba2d9d-8b02-4400-ae3b-c875100fcb68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.826727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba2d9d-8b02-4400-ae3b-c875100fcb68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.829060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-77ba2d9d-8b02-4400-ae3b-c875100fcb68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:00.851635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-9f957585-fba4-4a27-9bc3-76341863fedd None None] MFA Rules not processed for user `a971d893d9934e72b271a1a44d040ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:00.852870 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9f957585-fba4-4a27-9bc3-76341863fedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:00.863949 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:00.891389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba2d9d-8b02-4400-ae3b-c875100fcb68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.892960 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba2d9d-8b02-4400-ae3b-c875100fcb68 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.902039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1317/2652] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:19:00.918561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1336/2653] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:00.930024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba2d9d-8b02-4400-ae3b-c875100fcb68 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:00.931488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-77ba2d9d-8b02-4400-ae3b-c875100fcb68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:00.943347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba2d9d-8b02-4400-ae3b-c875100fcb68 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:00.947046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f010ef-c01b-4089-9c06-f98a293667d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:00.947647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f010ef-c01b-4089-9c06-f98a293667d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:00.947925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f010ef-c01b-4089-9c06-f98a293667d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:00.948197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f010ef-c01b-4089-9c06-f98a293667d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:00.956719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f010ef-c01b-4089-9c06-f98a293667d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:00.958134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f010ef-c01b-4089-9c06-f98a293667d0 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:00.968403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1318/2654] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:03.505687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f957585-fba4-4a27-9bc3-76341863fedd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:03.524473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:03.582691 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f957585-fba4-4a27-9bc3-76341863fedd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:03.587962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:03.588810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:03.589087 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:03.589361 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] PATH_INFO: `/v3/users/2ed8f99b53104681af7d148f88913e2f` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:03.590514 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed8f99b53104681af7d148f88913e2f)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:03.592449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:03.640231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed8f99b53104681af7d148f88913e2f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:03.651756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed8f99b53104681af7d148f88913e2f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:03.658651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed8f99b53104681af7d148f88913e2f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:03.676040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed8f99b53104681af7d148f88913e2f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:03.676821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed8f99b53104681af7d148f88913e2f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:03.684418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1337/2655] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:03 2022] DELETE /identity/v3/users/2ed8f99b53104681af7d148f88913e2f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:03.689410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6300935-81a2-4885-a9e0-14944df3f148 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:03.689921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6300935-81a2-4885-a9e0-14944df3f148 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:03.690030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6300935-81a2-4885-a9e0-14944df3f148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:03.710384 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f6300935-81a2-4885-a9e0-14944df3f148 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:03.711619 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f6300935-81a2-4885-a9e0-14944df3f148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:03.770601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1319/2656] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:19:03.780254 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:03.782389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:03.838173 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a084d9-4944-4e07-b4a6-d45eb76fb45a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:03.841812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24d4a5-e170-4bb7-8550-cba68052da73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:03.842411 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24d4a5-e170-4bb7-8550-cba68052da73 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:03.842714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24d4a5-e170-4bb7-8550-cba68052da73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:03.843009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24d4a5-e170-4bb7-8550-cba68052da73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:03.852618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24d4a5-e170-4bb7-8550-cba68052da73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:03.854029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24d4a5-e170-4bb7-8550-cba68052da73 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:03.863408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1338/2657] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:04.136403 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6300935-81a2-4885-a9e0-14944df3f148 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:04.137828 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f6300935-81a2-4885-a9e0-14944df3f148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:04.150350 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6300935-81a2-4885-a9e0-14944df3f148 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:04.155032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:04.155672 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:04.155961 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:04.156241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] PATH_INFO: `/v3/projects/a6daee4eb75742c2a917560bae62cede` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:04.157401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6daee4eb75742c2a917560bae62cede)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:04.159335 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:04.209014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6daee4eb75742c2a917560bae62cede', 'request_id': 'req-f3b36272-da00-4332-bbeb-946264112ebb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.214827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6daee4eb75742c2a917560bae62cede', 'request_id': 'req-f3b36272-da00-4332-bbeb-946264112ebb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.221822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6daee4eb75742c2a917560bae62cede', 'request_id': 'req-f3b36272-da00-4332-bbeb-946264112ebb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.234486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] The token cache is being invalidate because project a6daee4eb75742c2a917560bae62cede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:04.234796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.235455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.236944 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1320/2658] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:04 2022] DELETE /identity/v3/projects/a6daee4eb75742c2a917560bae62cede => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:04.240990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24d4a5-e170-4bb7-8550-cba68052da73 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:04.252373 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24d4a5-e170-4bb7-8550-cba68052da73 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:04.285675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:04.310680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5de80b-8b0c-4756-8697-82acd6ef854d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:04.311338 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5de80b-8b0c-4756-8697-82acd6ef854d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:04.311616 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5de80b-8b0c-4756-8697-82acd6ef854d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:04.311882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5de80b-8b0c-4756-8697-82acd6ef854d admin admin] PATH_INFO: `/v3/users/5836d2ac465f4032a6af48ca68d7ae7f` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:04.313746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5de80b-8b0c-4756-8697-82acd6ef854d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5836d2ac465f4032a6af48ca68d7ae7f)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:04.315655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5de80b-8b0c-4756-8697-82acd6ef854d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:04.362836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ce5de80b-8b0c-4756-8697-82acd6ef854d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5836d2ac465f4032a6af48ca68d7ae7f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.374357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ce5de80b-8b0c-4756-8697-82acd6ef854d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5836d2ac465f4032a6af48ca68d7ae7f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.381996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ce5de80b-8b0c-4756-8697-82acd6ef854d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5836d2ac465f4032a6af48ca68d7ae7f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.389025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ce5de80b-8b0c-4756-8697-82acd6ef854d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5836d2ac465f4032a6af48ca68d7ae7f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.389708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ce5de80b-8b0c-4756-8697-82acd6ef854d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5836d2ac465f4032a6af48ca68d7ae7f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.395863 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1339/2659] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:04 2022] DELETE /identity/v3/users/5836d2ac465f4032a6af48ca68d7ae7f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:04.669187 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:04.670626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:04.694556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b36272-da00-4332-bbeb-946264112ebb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:04.742391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:04.761258 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:04.761908 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:04.762171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:04.762442 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] PATH_INFO: `/v3/projects/925f0f6024de42518e0f00d3558210d9` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:04.763586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=925f0f6024de42518e0f00d3558210d9)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:04.765479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:04.813965 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '925f0f6024de42518e0f00d3558210d9', 'request_id': 'req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.819641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '925f0f6024de42518e0f00d3558210d9', 'request_id': 'req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.825825 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '925f0f6024de42518e0f00d3558210d9', 'request_id': 'req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.839060 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] The token cache is being invalidate because project 925f0f6024de42518e0f00d3558210d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:04.839352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.839973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fa94894f-f8f5-4c1b-8663-b7f8b93fbb15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:04.841464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1321/2660] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:04 2022] DELETE /identity/v3/projects/925f0f6024de42518e0f00d3558210d9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:04.846387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada39db-05d3-4b5c-b59e-1864d5f62f40 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:04.846904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada39db-05d3-4b5c-b59e-1864d5f62f40 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:04.847080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada39db-05d3-4b5c-b59e-1864d5f62f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:04.867382 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-9ada39db-05d3-4b5c-b59e-1864d5f62f40 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:04.868640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9ada39db-05d3-4b5c-b59e-1864d5f62f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:04.928233 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1340/2661] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:04.937061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbb998-dc45-473e-9b1c-10990a75694a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:04.937341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbb998-dc45-473e-9b1c-10990a75694a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:04.937616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbb998-dc45-473e-9b1c-10990a75694a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:04.959257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-93cbb998-dc45-473e-9b1c-10990a75694a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:04.960451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-93cbb998-dc45-473e-9b1c-10990a75694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:04.968665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1322/2662] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:04.973023 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada39db-05d3-4b5c-b59e-1864d5f62f40 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:04.982824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada39db-05d3-4b5c-b59e-1864d5f62f40 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:04.987516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d9874-2597-451e-825a-7e7545c3e718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:04.988114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d9874-2597-451e-825a-7e7545c3e718 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:04.988392 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d9874-2597-451e-825a-7e7545c3e718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:04.988661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d9874-2597-451e-825a-7e7545c3e718 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:04.989552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d9874-2597-451e-825a-7e7545c3e718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:19:04.989859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d9874-2597-451e-825a-7e7545c3e718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:04.991975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d9874-2597-451e-825a-7e7545c3e718 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.004835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1341/2663] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:19:05.009293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbb998-dc45-473e-9b1c-10990a75694a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.019100 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbb998-dc45-473e-9b1c-10990a75694a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.023765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d784e-50cf-45f6-a5aa-30bb977915c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.024363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d784e-50cf-45f6-a5aa-30bb977915c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.024645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d784e-50cf-45f6-a5aa-30bb977915c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.024920 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d784e-50cf-45f6-a5aa-30bb977915c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.025552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d784e-50cf-45f6-a5aa-30bb977915c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:05.027479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d784e-50cf-45f6-a5aa-30bb977915c1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.054233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1323/2664] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:05 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:05.058506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d9874-2597-451e-825a-7e7545c3e718 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.069331 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d9874-2597-451e-825a-7e7545c3e718 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.113436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:05.130437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056671f-14d8-4353-931a-791444529ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.131067 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056671f-14d8-4353-931a-791444529ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.131346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056671f-14d8-4353-931a-791444529ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.131614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056671f-14d8-4353-931a-791444529ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.132251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056671f-14d8-4353-931a-791444529ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:05.134131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056671f-14d8-4353-931a-791444529ae7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.156073 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1342/2665] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:05 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:19:05.161569 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d784e-50cf-45f6-a5aa-30bb977915c1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.171951 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d784e-50cf-45f6-a5aa-30bb977915c1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.177364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774ae29-5b17-4c62-8ec1-2ab8d15b7bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.177963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774ae29-5b17-4c62-8ec1-2ab8d15b7bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.178243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774ae29-5b17-4c62-8ec1-2ab8d15b7bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.178519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774ae29-5b17-4c62-8ec1-2ab8d15b7bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.179192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774ae29-5b17-4c62-8ec1-2ab8d15b7bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:05.180781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774ae29-5b17-4c62-8ec1-2ab8d15b7bd7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.191598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1324/2666] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:05.195627 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056671f-14d8-4353-931a-791444529ae7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.205677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056671f-14d8-4353-931a-791444529ae7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.210264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5653189-69a9-4bce-8cbf-72ab47e6d3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.210853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5653189-69a9-4bce-8cbf-72ab47e6d3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.211170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5653189-69a9-4bce-8cbf-72ab47e6d3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.211441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5653189-69a9-4bce-8cbf-72ab47e6d3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.212100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5653189-69a9-4bce-8cbf-72ab47e6d3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:05.213660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5653189-69a9-4bce-8cbf-72ab47e6d3ba admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.223881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1343/2667] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:05.228010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774ae29-5b17-4c62-8ec1-2ab8d15b7bd7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.238815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774ae29-5b17-4c62-8ec1-2ab8d15b7bd7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.243446 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae4346-1c96-46c4-bfd0-d5e833481fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.244095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae4346-1c96-46c4-bfd0-d5e833481fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.244374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae4346-1c96-46c4-bfd0-d5e833481fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.244645 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae4346-1c96-46c4-bfd0-d5e833481fc8 admin admin] PATH_INFO: `/v3/projects/01b02fad57f24ce18f81cb861cbb48a5/users/d89542bbec38490e8562763567486a65/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.261120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae4346-1c96-46c4-bfd0-d5e833481fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b02fad57f24ce18f81cb861cbb48a5, user_id=d89542bbec38490e8562763567486a65, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:05.263765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae4346-1c96-46c4-bfd0-d5e833481fc8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.277162 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1325/2668] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:05 2022] PUT /identity/v3/projects/01b02fad57f24ce18f81cb861cbb48a5/users/d89542bbec38490e8562763567486a65/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:05.287964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c5580-1b88-44fb-bb25-d1cd629c5954 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.288253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c5580-1b88-44fb-bb25-d1cd629c5954 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.288554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c5580-1b88-44fb-bb25-d1cd629c5954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.319215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-221c5580-1b88-44fb-bb25-d1cd629c5954 None None] MFA Rules not processed for user `d89542bbec38490e8562763567486a65`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:05.320445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-221c5580-1b88-44fb-bb25-d1cd629c5954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:05.383377 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1344/2669] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:19:05.387899 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae4346-1c96-46c4-bfd0-d5e833481fc8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.448296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae4346-1c96-46c4-bfd0-d5e833481fc8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.452977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84905f59-bd76-4719-900c-c31fea7750a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.454082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84905f59-bd76-4719-900c-c31fea7750a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.454372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84905f59-bd76-4719-900c-c31fea7750a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.454663 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84905f59-bd76-4719-900c-c31fea7750a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.455314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84905f59-bd76-4719-900c-c31fea7750a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:05.457192 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84905f59-bd76-4719-900c-c31fea7750a6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.472460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:05.486501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1326/2670] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:05 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:05.490546 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c5580-1b88-44fb-bb25-d1cd629c5954 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.501545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c5580-1b88-44fb-bb25-d1cd629c5954 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.558614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cdec7-8ac9-400f-8467-71c51b5a7fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.559170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cdec7-8ac9-400f-8467-71c51b5a7fde admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.559455 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cdec7-8ac9-400f-8467-71c51b5a7fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.559730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cdec7-8ac9-400f-8467-71c51b5a7fde admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.560371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cdec7-8ac9-400f-8467-71c51b5a7fde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:05.562287 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cdec7-8ac9-400f-8467-71c51b5a7fde admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.584390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1345/2671] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:05 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:05.588573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84905f59-bd76-4719-900c-c31fea7750a6 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.599001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84905f59-bd76-4719-900c-c31fea7750a6 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.603610 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6052ec1-06f6-487b-bafa-6d714db8d0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.604228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6052ec1-06f6-487b-bafa-6d714db8d0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.604503 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6052ec1-06f6-487b-bafa-6d714db8d0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.604770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6052ec1-06f6-487b-bafa-6d714db8d0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.605398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6052ec1-06f6-487b-bafa-6d714db8d0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:05.607030 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6052ec1-06f6-487b-bafa-6d714db8d0d7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.617381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1327/2672] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:05.621440 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cdec7-8ac9-400f-8467-71c51b5a7fde admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.633110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cdec7-8ac9-400f-8467-71c51b5a7fde admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.638070 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66b2c4-6e65-4194-bf57-8676e1410375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.638838 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66b2c4-6e65-4194-bf57-8676e1410375 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.639148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66b2c4-6e65-4194-bf57-8676e1410375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.639428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66b2c4-6e65-4194-bf57-8676e1410375 admin admin] PATH_INFO: `/v3/projects/583ee4e1c1b1408aae1c80992f071135/users/2868354f157143529bf1f44c954098b2/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.658238 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66b2c4-6e65-4194-bf57-8676e1410375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=583ee4e1c1b1408aae1c80992f071135, user_id=2868354f157143529bf1f44c954098b2, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:05.660880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66b2c4-6e65-4194-bf57-8676e1410375 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.673898 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1346/2673] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:05 2022] PUT /identity/v3/projects/583ee4e1c1b1408aae1c80992f071135/users/2868354f157143529bf1f44c954098b2/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:05.678263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6052ec1-06f6-487b-bafa-6d714db8d0d7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.689272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6052ec1-06f6-487b-bafa-6d714db8d0d7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.748919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa90fd-c459-4252-b1ef-a4859e8cfa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.749529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa90fd-c459-4252-b1ef-a4859e8cfa4b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.749808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa90fd-c459-4252-b1ef-a4859e8cfa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.750081 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa90fd-c459-4252-b1ef-a4859e8cfa4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.751272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa90fd-c459-4252-b1ef-a4859e8cfa4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:05.752873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa90fd-c459-4252-b1ef-a4859e8cfa4b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.762926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1328/2674] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:05.767624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66b2c4-6e65-4194-bf57-8676e1410375 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.777755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66b2c4-6e65-4194-bf57-8676e1410375 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.782353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e85a1e-4d86-4b1c-8c13-b451b9d5648b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.783019 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85a1e-4d86-4b1c-8c13-b451b9d5648b admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.783303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85a1e-4d86-4b1c-8c13-b451b9d5648b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.783573 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85a1e-4d86-4b1c-8c13-b451b9d5648b admin admin] PATH_INFO: `/v3/projects/583ee4e1c1b1408aae1c80992f071135/users/2868354f157143529bf1f44c954098b2/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.785525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e85a1e-4d86-4b1c-8c13-b451b9d5648b admin admin] RBAC: Authorizing `identity:create_grant(project_id=583ee4e1c1b1408aae1c80992f071135, user_id=2868354f157143529bf1f44c954098b2, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:05.788153 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e85a1e-4d86-4b1c-8c13-b451b9d5648b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:05.798235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1347/2675] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:05 2022] PUT /identity/v3/projects/583ee4e1c1b1408aae1c80992f071135/users/2868354f157143529bf1f44c954098b2/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:05.808490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e5aad-d70f-4425-9b0a-adb3f9b55bde None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.808835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e5aad-d70f-4425-9b0a-adb3f9b55bde None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.809065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e5aad-d70f-4425-9b0a-adb3f9b55bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.837313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-078e5aad-d70f-4425-9b0a-adb3f9b55bde None None] MFA Rules not processed for user `2868354f157143529bf1f44c954098b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:05.838519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-078e5aad-d70f-4425-9b0a-adb3f9b55bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:05.909909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1329/2676] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:05.920523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e85a1e-4d86-4b1c-8c13-b451b9d5648b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:05.922746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e8e85a1e-4d86-4b1c-8c13-b451b9d5648b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:05.929544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:05.981561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e85a1e-4d86-4b1c-8c13-b451b9d5648b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:05.985791 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ecd78-b9b6-4fdf-b81e-e3daac821ae9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:05.986386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ecd78-b9b6-4fdf-b81e-e3daac821ae9 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:05.986664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ecd78-b9b6-4fdf-b81e-e3daac821ae9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:05.986933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ecd78-b9b6-4fdf-b81e-e3daac821ae9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:05.999094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ecd78-b9b6-4fdf-b81e-e3daac821ae9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:06.001926 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ecd78-b9b6-4fdf-b81e-e3daac821ae9 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:06.014180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1348/2677] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:06.388777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e5aad-d70f-4425-9b0a-adb3f9b55bde None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:06.450936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e5aad-d70f-4425-9b0a-adb3f9b55bde None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:06.456227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:06.456857 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:06.457137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:06.457409 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] PATH_INFO: `/v3/users/d89542bbec38490e8562763567486a65` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:06.458579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89542bbec38490e8562763567486a65)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:06.460522 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:06.465941 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:06.506257 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89542bbec38490e8562763567486a65'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:06.518356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89542bbec38490e8562763567486a65'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:06.525492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89542bbec38490e8562763567486a65'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:06.526180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89542bbec38490e8562763567486a65'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:06.534059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89542bbec38490e8562763567486a65'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:06.541127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1330/2678] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:06 2022] DELETE /identity/v3/users/d89542bbec38490e8562763567486a65 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:06.545404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09c9bb-1c78-4d5a-949c-f41d6809b0a0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:06.545690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09c9bb-1c78-4d5a-949c-f41d6809b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:06.545957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09c9bb-1c78-4d5a-949c-f41d6809b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:06.565097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ee09c9bb-1c78-4d5a-949c-f41d6809b0a0 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:06.566219 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ee09c9bb-1c78-4d5a-949c-f41d6809b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:06.625910 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1349/2679] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:06.635499 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:06.636894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:06.695528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1e998-9605-4a56-8025-08333177c7ec admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:06.699181 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c0ee6-fed9-45d4-8233-817b77f9a4fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:06.699790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c0ee6-fed9-45d4-8233-817b77f9a4fe service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:06.700067 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c0ee6-fed9-45d4-8233-817b77f9a4fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:06.700340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c0ee6-fed9-45d4-8233-817b77f9a4fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:06.709354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c0ee6-fed9-45d4-8233-817b77f9a4fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:06.710778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c0ee6-fed9-45d4-8233-817b77f9a4fe service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:06.720174 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1331/2680] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:06.985887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09c9bb-1c78-4d5a-949c-f41d6809b0a0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:06.987454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ee09c9bb-1c78-4d5a-949c-f41d6809b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:06.999444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09c9bb-1c78-4d5a-949c-f41d6809b0a0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:07.005034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:07.005672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:07.005953 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:07.006294 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] PATH_INFO: `/v3/projects/01b02fad57f24ce18f81cb861cbb48a5` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:07.007817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b02fad57f24ce18f81cb861cbb48a5)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:07.009365 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:07.059896 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b02fad57f24ce18f81cb861cbb48a5', 'request_id': 'req-3ebde730-affe-42f9-a543-20155192704c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.066552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b02fad57f24ce18f81cb861cbb48a5', 'request_id': 'req-3ebde730-affe-42f9-a543-20155192704c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.072235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b02fad57f24ce18f81cb861cbb48a5', 'request_id': 'req-3ebde730-affe-42f9-a543-20155192704c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.084920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] The token cache is being invalidate because project 01b02fad57f24ce18f81cb861cbb48a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:07.085210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.085843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.087332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1350/2681] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:06 2022] DELETE /identity/v3/projects/01b02fad57f24ce18f81cb861cbb48a5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:07.091812 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c0ee6-fed9-45d4-8233-817b77f9a4fe service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:07.101510 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c0ee6-fed9-45d4-8233-817b77f9a4fe service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:07.157465 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0e585-b62e-4208-bfc0-ceec40531b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:07.158084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0e585-b62e-4208-bfc0-ceec40531b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:07.158359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0e585-b62e-4208-bfc0-ceec40531b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:07.158626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0e585-b62e-4208-bfc0-ceec40531b46 admin admin] PATH_INFO: `/v3/users/2868354f157143529bf1f44c954098b2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:07.159776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0e585-b62e-4208-bfc0-ceec40531b46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2868354f157143529bf1f44c954098b2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:07.161682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0e585-b62e-4208-bfc0-ceec40531b46 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:07.194612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:07.210280 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-43b0e585-b62e-4208-bfc0-ceec40531b46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2868354f157143529bf1f44c954098b2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.222197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-43b0e585-b62e-4208-bfc0-ceec40531b46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2868354f157143529bf1f44c954098b2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.229134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-43b0e585-b62e-4208-bfc0-ceec40531b46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2868354f157143529bf1f44c954098b2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.229830 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-43b0e585-b62e-4208-bfc0-ceec40531b46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2868354f157143529bf1f44c954098b2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.237726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-43b0e585-b62e-4208-bfc0-ceec40531b46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2868354f157143529bf1f44c954098b2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.244215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1332/2682] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:07 2022] DELETE /identity/v3/users/2868354f157143529bf1f44c954098b2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:07.526453 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:07.528177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:07.543284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebde730-affe-42f9-a543-20155192704c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:07.574356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:07.605138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:07.605692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:07.605969 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:07.606237 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] PATH_INFO: `/v3/projects/583ee4e1c1b1408aae1c80992f071135` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:07.607382 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=583ee4e1c1b1408aae1c80992f071135)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:07.609295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:07.656881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583ee4e1c1b1408aae1c80992f071135', 'request_id': 'req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.663516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583ee4e1c1b1408aae1c80992f071135', 'request_id': 'req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.669236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583ee4e1c1b1408aae1c80992f071135', 'request_id': 'req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.684047 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] The token cache is being invalidate because project 583ee4e1c1b1408aae1c80992f071135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:07.684219 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.684996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-17903014-f6ed-48d8-9ab9-6a05b8fd4da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:07.686499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1351/2683] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:07 2022] DELETE /identity/v3/projects/583ee4e1c1b1408aae1c80992f071135 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:07.692671 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9847df-dcb5-4da7-8975-247c54fce84d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:07.692988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9847df-dcb5-4da7-8975-247c54fce84d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:07.693188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9847df-dcb5-4da7-8975-247c54fce84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:07.713207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3a9847df-dcb5-4da7-8975-247c54fce84d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:07.714417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3a9847df-dcb5-4da7-8975-247c54fce84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:07.773888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1333/2684] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:07.781462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cba908-edcc-49c0-982b-40817d0bee9f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:07.782231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cba908-edcc-49c0-982b-40817d0bee9f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:07.782502 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cba908-edcc-49c0-982b-40817d0bee9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:07.802526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f6cba908-edcc-49c0-982b-40817d0bee9f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:07.803729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f6cba908-edcc-49c0-982b-40817d0bee9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:07.811413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1352/2685] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:07.815549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9847df-dcb5-4da7-8975-247c54fce84d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:07.825316 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9847df-dcb5-4da7-8975-247c54fce84d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:07.829839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ccf58-925b-4366-9e77-3e0a9a0c9985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:07.830432 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ccf58-925b-4366-9e77-3e0a9a0c9985 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:07.830712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ccf58-925b-4366-9e77-3e0a9a0c9985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:07.831015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ccf58-925b-4366-9e77-3e0a9a0c9985 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:07.831660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ccf58-925b-4366-9e77-3e0a9a0c9985 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:19:07.831963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ccf58-925b-4366-9e77-3e0a9a0c9985 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:07.833559 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ccf58-925b-4366-9e77-3e0a9a0c9985 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:07.845397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1334/2686] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:19:07.849560 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cba908-edcc-49c0-982b-40817d0bee9f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:07.858937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cba908-edcc-49c0-982b-40817d0bee9f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:07.864162 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdc533-17ed-4f90-9ed8-a639e730e252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:07.864753 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdc533-17ed-4f90-9ed8-a639e730e252 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:07.865033 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdc533-17ed-4f90-9ed8-a639e730e252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:07.865311 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdc533-17ed-4f90-9ed8-a639e730e252 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:07.865947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbdc533-17ed-4f90-9ed8-a639e730e252 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:07.867849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbdc533-17ed-4f90-9ed8-a639e730e252 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:07.895204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1353/2687] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:07 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:19:07.899635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ccf58-925b-4366-9e77-3e0a9a0c9985 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:07.910097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ccf58-925b-4366-9e77-3e0a9a0c9985 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:08.096639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d18167-fe4c-4ba2-84be-ecba82641cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:08.097329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d18167-fe4c-4ba2-84be-ecba82641cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.097507 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d18167-fe4c-4ba2-84be-ecba82641cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.097778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d18167-fe4c-4ba2-84be-ecba82641cd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.098492 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d18167-fe4c-4ba2-84be-ecba82641cd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:08.100368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d18167-fe4c-4ba2-84be-ecba82641cd9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:08.121549 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1335/2688] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:07 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:08.125790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdc533-17ed-4f90-9ed8-a639e730e252 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:08.135955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdc533-17ed-4f90-9ed8-a639e730e252 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:08.140436 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9927dc-a83f-4e51-b83b-4549cd578fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:08.141547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9927dc-a83f-4e51-b83b-4549cd578fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.141827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9927dc-a83f-4e51-b83b-4549cd578fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.142100 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9927dc-a83f-4e51-b83b-4549cd578fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.142738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9927dc-a83f-4e51-b83b-4549cd578fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:08.144431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9927dc-a83f-4e51-b83b-4549cd578fa4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:08.154166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1354/2689] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:08.158691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d18167-fe4c-4ba2-84be-ecba82641cd9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:08.168500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d18167-fe4c-4ba2-84be-ecba82641cd9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:08.173590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18141449-f9eb-4c03-b275-fb4e23d5254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:08.174180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141449-f9eb-4c03-b275-fb4e23d5254c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.174458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141449-f9eb-4c03-b275-fb4e23d5254c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.175014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141449-f9eb-4c03-b275-fb4e23d5254c admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.175658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18141449-f9eb-4c03-b275-fb4e23d5254c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:08.177223 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18141449-f9eb-4c03-b275-fb4e23d5254c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:08.187076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1336/2690] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:08.191803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9927dc-a83f-4e51-b83b-4549cd578fa4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:08.203106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9927dc-a83f-4e51-b83b-4549cd578fa4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:08.207776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bec08f-4ea6-4ab3-99e5-76426143620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:08.208430 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bec08f-4ea6-4ab3-99e5-76426143620a admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.208707 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bec08f-4ea6-4ab3-99e5-76426143620a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.208975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bec08f-4ea6-4ab3-99e5-76426143620a admin admin] PATH_INFO: `/v3/projects/9e16a653e64b4c0aa739a877fa73bf76/users/d54c900e2947403ebb384282b33385cb/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.223660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bec08f-4ea6-4ab3-99e5-76426143620a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e16a653e64b4c0aa739a877fa73bf76, user_id=d54c900e2947403ebb384282b33385cb, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:08.226271 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bec08f-4ea6-4ab3-99e5-76426143620a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:08.239989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1355/2691] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:08 2022] PUT /identity/v3/projects/9e16a653e64b4c0aa739a877fa73bf76/users/d54c900e2947403ebb384282b33385cb/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:08.250277 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d92b0c-3870-453e-b667-3250ae608876 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.251368 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d92b0c-3870-453e-b667-3250ae608876 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.251489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d92b0c-3870-453e-b667-3250ae608876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.276562 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:08.280088 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-02d92b0c-3870-453e-b667-3250ae608876 None None] MFA Rules not processed for user `d54c900e2947403ebb384282b33385cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:08.281801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-02d92b0c-3870-453e-b667-3250ae608876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:08.348820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1337/2692] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:08.354166 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bec08f-4ea6-4ab3-99e5-76426143620a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:08.412095 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:08.415951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bec08f-4ea6-4ab3-99e5-76426143620a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:08.421964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e064f-61c3-4b0b-9c15-f62232164d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:08.422753 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e064f-61c3-4b0b-9c15-f62232164d02 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.423208 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e064f-61c3-4b0b-9c15-f62232164d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.423534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e064f-61c3-4b0b-9c15-f62232164d02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.424342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e064f-61c3-4b0b-9c15-f62232164d02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:08.426528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e064f-61c3-4b0b-9c15-f62232164d02 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:08.452297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1356/2693] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:08 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:08.456859 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d92b0c-3870-453e-b667-3250ae608876 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:08.467348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d92b0c-3870-453e-b667-3250ae608876 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:08.523724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141ba62-3e5d-4038-9e69-4116c26d2c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:08.524317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141ba62-3e5d-4038-9e69-4116c26d2c31 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.524592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141ba62-3e5d-4038-9e69-4116c26d2c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.524863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141ba62-3e5d-4038-9e69-4116c26d2c31 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.525475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141ba62-3e5d-4038-9e69-4116c26d2c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:08.527390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141ba62-3e5d-4038-9e69-4116c26d2c31 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:08.551193 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1338/2694] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:08 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:08.555144 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e064f-61c3-4b0b-9c15-f62232164d02 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:08.565945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e064f-61c3-4b0b-9c15-f62232164d02 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:08.570614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7c57a-8ca3-457e-a280-803a967d5bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:08.571238 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c57a-8ca3-457e-a280-803a967d5bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.571520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c57a-8ca3-457e-a280-803a967d5bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.571793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c57a-8ca3-457e-a280-803a967d5bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.572461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7c57a-8ca3-457e-a280-803a967d5bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:08.574055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7c57a-8ca3-457e-a280-803a967d5bf0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:08.586421 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1357/2695] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:08.590489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141ba62-3e5d-4038-9e69-4116c26d2c31 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:08.600928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141ba62-3e5d-4038-9e69-4116c26d2c31 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:08.605772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078961a-0a42-498c-9646-6c070e3a6a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:08.606372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078961a-0a42-498c-9646-6c070e3a6a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.606691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078961a-0a42-498c-9646-6c070e3a6a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.607011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078961a-0a42-498c-9646-6c070e3a6a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.607646 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078961a-0a42-498c-9646-6c070e3a6a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:08.609892 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078961a-0a42-498c-9646-6c070e3a6a5a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:08.619727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1339/2696] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:08.624055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7c57a-8ca3-457e-a280-803a967d5bf0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:08.634825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7c57a-8ca3-457e-a280-803a967d5bf0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:08.639498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914c804-6345-468d-a7ef-3306688a7d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:08.640136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914c804-6345-468d-a7ef-3306688a7d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.640414 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914c804-6345-468d-a7ef-3306688a7d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.640693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914c804-6345-468d-a7ef-3306688a7d91 admin admin] PATH_INFO: `/v3/projects/bcb2e6ec21004824b3b935421a89e77d/users/42b1791bea284110bce277f43e11d0e1/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.655645 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914c804-6345-468d-a7ef-3306688a7d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb2e6ec21004824b3b935421a89e77d, user_id=42b1791bea284110bce277f43e11d0e1, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:08.658248 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914c804-6345-468d-a7ef-3306688a7d91 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:08.671587 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1358/2697] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:08 2022] PUT /identity/v3/projects/bcb2e6ec21004824b3b935421a89e77d/users/42b1791bea284110bce277f43e11d0e1/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:08.741475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc2d2a-63be-4249-8067-a351d98ef26d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.741824 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc2d2a-63be-4249-8067-a351d98ef26d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.742189 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc2d2a-63be-4249-8067-a351d98ef26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.774184 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-aefc2d2a-63be-4249-8067-a351d98ef26d None None] MFA Rules not processed for user `42b1791bea284110bce277f43e11d0e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:08.775418 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-aefc2d2a-63be-4249-8067-a351d98ef26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:08.851621 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1340/2698] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:08.856180 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914c804-6345-468d-a7ef-3306688a7d91 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:08.925527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914c804-6345-468d-a7ef-3306688a7d91 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:08.930368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e25bf0-3ecf-4c29-ae61-75df31303913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:08.930997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25bf0-3ecf-4c29-ae61-75df31303913 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:08.931279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25bf0-3ecf-4c29-ae61-75df31303913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:08.931547 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25bf0-3ecf-4c29-ae61-75df31303913 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:08.932195 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e25bf0-3ecf-4c29-ae61-75df31303913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:08.934079 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e25bf0-3ecf-4c29-ae61-75df31303913 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:08.969477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1359/2699] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:08 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:08.970450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc2d2a-63be-4249-8067-a351d98ef26d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:08.981819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc2d2a-63be-4249-8067-a351d98ef26d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:08.991252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:09.046772 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6aa4d-cf4c-43a9-b94c-5e904bf96b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:09.047935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6aa4d-cf4c-43a9-b94c-5e904bf96b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:09.048213 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6aa4d-cf4c-43a9-b94c-5e904bf96b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:09.048504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6aa4d-cf4c-43a9-b94c-5e904bf96b31 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:09.049139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6aa4d-cf4c-43a9-b94c-5e904bf96b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:09.051434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6aa4d-cf4c-43a9-b94c-5e904bf96b31 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:09.075611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1341/2700] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:08 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:19:09.079619 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e25bf0-3ecf-4c29-ae61-75df31303913 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:09.090812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e25bf0-3ecf-4c29-ae61-75df31303913 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:09.098210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ccc64-eff2-48a0-8472-b1ea1061b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:09.099267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ccc64-eff2-48a0-8472-b1ea1061b6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:09.099706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6aa4d-cf4c-43a9-b94c-5e904bf96b31 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:09.100168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ccc64-eff2-48a0-8472-b1ea1061b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:09.100333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ccc64-eff2-48a0-8472-b1ea1061b6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:09.100982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ccc64-eff2-48a0-8472-b1ea1061b6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:09.102025 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-46b6aa4d-cf4c-43a9-b94c-5e904bf96b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:09.103941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ccc64-eff2-48a0-8472-b1ea1061b6a9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:09.114987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1360/2701] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:09.118900 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ccc64-eff2-48a0-8472-b1ea1061b6a9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:09.130149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ccc64-eff2-48a0-8472-b1ea1061b6a9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:09.135593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f02009-df6e-469f-bd04-5aba3e4d7468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:09.136400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f02009-df6e-469f-bd04-5aba3e4d7468 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:09.136735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f02009-df6e-469f-bd04-5aba3e4d7468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:09.137084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f02009-df6e-469f-bd04-5aba3e4d7468 admin admin] PATH_INFO: `/v3/projects/99b7adc3f8324d1c9e673456643ca4e8/users/80265799eb704a4fb6728e68506f022c/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:09.155770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f02009-df6e-469f-bd04-5aba3e4d7468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b7adc3f8324d1c9e673456643ca4e8, user_id=80265799eb704a4fb6728e68506f022c, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:09.160059 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f02009-df6e-469f-bd04-5aba3e4d7468 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:09.176222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1361/2702] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:09 2022] PUT /identity/v3/projects/99b7adc3f8324d1c9e673456643ca4e8/users/80265799eb704a4fb6728e68506f022c/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:09.177742 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6aa4d-cf4c-43a9-b94c-5e904bf96b31 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:09.179910 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f02009-df6e-469f-bd04-5aba3e4d7468 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:09.245579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f24977-0c4d-4d86-9f89-1053a2b88a67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:09.246293 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24977-0c4d-4d86-9f89-1053a2b88a67 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:09.246469 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24977-0c4d-4d86-9f89-1053a2b88a67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:09.246723 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24977-0c4d-4d86-9f89-1053a2b88a67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:09.248535 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d8f24977-0c4d-4d86-9f89-1053a2b88a67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:09.256879 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:09.261765 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f02009-df6e-469f-bd04-5aba3e4d7468 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:09.272639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae465f1f-05da-40f7-8135-b3aa3b41cfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:09.273465 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae465f1f-05da-40f7-8135-b3aa3b41cfb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:09.273968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae465f1f-05da-40f7-8135-b3aa3b41cfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:09.274331 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae465f1f-05da-40f7-8135-b3aa3b41cfb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:09.275116 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae465f1f-05da-40f7-8135-b3aa3b41cfb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:09.276921 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae465f1f-05da-40f7-8135-b3aa3b41cfb6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:09.303453 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1362/2703] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:09.308667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae465f1f-05da-40f7-8135-b3aa3b41cfb6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:09.320717 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae465f1f-05da-40f7-8135-b3aa3b41cfb6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:09.326268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f76058-3113-4848-be04-ba34dcaa2a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:09.327131 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f76058-3113-4848-be04-ba34dcaa2a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:09.327368 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f76058-3113-4848-be04-ba34dcaa2a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:09.327862 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f76058-3113-4848-be04-ba34dcaa2a42 admin admin] PATH_INFO: `/v3/projects/99b7adc3f8324d1c9e673456643ca4e8/users/80265799eb704a4fb6728e68506f022c/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:09.331142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f76058-3113-4848-be04-ba34dcaa2a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b7adc3f8324d1c9e673456643ca4e8, user_id=80265799eb704a4fb6728e68506f022c, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:09.337937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f76058-3113-4848-be04-ba34dcaa2a42 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:09.347486 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1363/2704] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:09 2022] PUT /identity/v3/projects/99b7adc3f8324d1c9e673456643ca4e8/users/80265799eb704a4fb6728e68506f022c/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:09.357025 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c94e9-f856-4851-b63c-bfd7c84f6464 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:09.357312 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c94e9-f856-4851-b63c-bfd7c84f6464 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:09.357595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c94e9-f856-4851-b63c-bfd7c84f6464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:09.358639 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:09.372760 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f24977-0c4d-4d86-9f89-1053a2b88a67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:09.374196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f24977-0c4d-4d86-9f89-1053a2b88a67 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:09.386584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-fe9c94e9-f856-4851-b63c-bfd7c84f6464 None None] MFA Rules not processed for user `80265799eb704a4fb6728e68506f022c`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:09.387830 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fe9c94e9-f856-4851-b63c-bfd7c84f6464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:09.398641 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1342/2705] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:19:09.471869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1364/2706] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:19:09.486753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f24977-0c4d-4d86-9f89-1053a2b88a67 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:09.488512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d8f24977-0c4d-4d86-9f89-1053a2b88a67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:09.500046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f24977-0c4d-4d86-9f89-1053a2b88a67 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:09.554997 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0897-90b2-4067-be15-09b10080f17b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:09.555675 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0897-90b2-4067-be15-09b10080f17b service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:09.555958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0897-90b2-4067-be15-09b10080f17b service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:09.556241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0897-90b2-4067-be15-09b10080f17b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:09.620873 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a0897-90b2-4067-be15-09b10080f17b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:09.622317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a0897-90b2-4067-be15-09b10080f17b service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:09.632885 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1343/2707] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:11.194261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c94e9-f856-4851-b63c-bfd7c84f6464 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:11.204472 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c94e9-f856-4851-b63c-bfd7c84f6464 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:11.208133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581a255-5381-44ed-a097-cfb4a88e520b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:11.208735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581a255-5381-44ed-a097-cfb4a88e520b service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:11.209016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581a255-5381-44ed-a097-cfb4a88e520b service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:11.209293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581a255-5381-44ed-a097-cfb4a88e520b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:11.269301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581a255-5381-44ed-a097-cfb4a88e520b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:11.270696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581a255-5381-44ed-a097-cfb4a88e520b service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:11.280676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1365/2708] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:11.582635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0897-90b2-4067-be15-09b10080f17b service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:11.584225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-459a0897-90b2-4067-be15-09b10080f17b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:11.652624 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0897-90b2-4067-be15-09b10080f17b service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:11.657146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:11.657797 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:11.658069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:11.658338 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] PATH_INFO: `/v3/users/a971d893d9934e72b271a1a44d040ad4` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:11.659700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a971d893d9934e72b271a1a44d040ad4)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:11.661561 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:11.703421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:11.712442 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a971d893d9934e72b271a1a44d040ad4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:11.728718 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a971d893d9934e72b271a1a44d040ad4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:11.736613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a971d893d9934e72b271a1a44d040ad4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:11.737372 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a971d893d9934e72b271a1a44d040ad4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:11.745837 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a971d893d9934e72b271a1a44d040ad4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:11.753798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1344/2709] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:11 2022] DELETE /identity/v3/users/a971d893d9934e72b271a1a44d040ad4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:11.758779 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3690b-f28d-4dc5-a349-2bd7e432c2a7 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:11.759417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3690b-f28d-4dc5-a349-2bd7e432c2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:11.759835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3690b-f28d-4dc5-a349-2bd7e432c2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:11.785877 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-40d3690b-f28d-4dc5-a349-2bd7e432c2a7 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:11.787136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-40d3690b-f28d-4dc5-a349-2bd7e432c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:11.854083 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1366/2710] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:11.854468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:11.856146 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:11.862762 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3690b-f28d-4dc5-a349-2bd7e432c2a7 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:11.864278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-40d3690b-f28d-4dc5-a349-2bd7e432c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:11.870890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:11.922975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd6dc0-baae-4544-90df-f45365b55833 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:11.927211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae77e1-2c7d-4ee1-a422-b888afde7dde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:11.927910 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae77e1-2c7d-4ee1-a422-b888afde7dde service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:11.928188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae77e1-2c7d-4ee1-a422-b888afde7dde service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:11.928458 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae77e1-2c7d-4ee1-a422-b888afde7dde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:11.928609 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3690b-f28d-4dc5-a349-2bd7e432c2a7 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:11.930235 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-feae77e1-2c7d-4ee1-a422-b888afde7dde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:11.932466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2498ac-8962-4d5d-bd0c-93b0ac12e1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:11.933062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2498ac-8962-4d5d-bd0c-93b0ac12e1ee service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:11.933337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2498ac-8962-4d5d-bd0c-93b0ac12e1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:11.933605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2498ac-8962-4d5d-bd0c-93b0ac12e1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:11.944010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2498ac-8962-4d5d-bd0c-93b0ac12e1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:11.945433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2498ac-8962-4d5d-bd0c-93b0ac12e1ee service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:11.959159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1367/2711] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:19:12.008568 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae77e1-2c7d-4ee1-a422-b888afde7dde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:12.010804 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae77e1-2c7d-4ee1-a422-b888afde7dde service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:12.024732 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1345/2712] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:19:12.285031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2498ac-8962-4d5d-bd0c-93b0ac12e1ee service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:12.286483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3a2498ac-8962-4d5d-bd0c-93b0ac12e1ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:12.304362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2498ac-8962-4d5d-bd0c-93b0ac12e1ee service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:12.310747 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:12.311481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:12.311999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:12.312328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] PATH_INFO: `/v3/projects/60b72c76d808413894f0fbd90fbaf499` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:12.314132 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b72c76d808413894f0fbd90fbaf499)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:12.316477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:12.367152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b72c76d808413894f0fbd90fbaf499', 'request_id': 'req-0b527096-1ecb-4277-89f7-4ac3a737071b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:12.373390 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b72c76d808413894f0fbd90fbaf499', 'request_id': 'req-0b527096-1ecb-4277-89f7-4ac3a737071b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:12.380877 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b72c76d808413894f0fbd90fbaf499', 'request_id': 'req-0b527096-1ecb-4277-89f7-4ac3a737071b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:12.398159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] The token cache is being invalidate because project 60b72c76d808413894f0fbd90fbaf499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:12.398459 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:12.399139 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0b527096-1ecb-4277-89f7-4ac3a737071b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:12.415261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1368/2713] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:12 2022] DELETE /identity/v3/projects/60b72c76d808413894f0fbd90fbaf499 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:12.415411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a91b4-1634-4877-89c5-a321c7103935 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:12.415411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a91b4-1634-4877-89c5-a321c7103935 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:12.415411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a91b4-1634-4877-89c5-a321c7103935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:12.432642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b81a91b4-1634-4877-89c5-a321c7103935 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:12.433857 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b81a91b4-1634-4877-89c5-a321c7103935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:12.503407 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1346/2714] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:12.511360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454555f-0931-4bc6-b657-9e34096c9dca None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:12.511644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454555f-0931-4bc6-b657-9e34096c9dca None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:12.511914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454555f-0931-4bc6-b657-9e34096c9dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:12.565598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e454555f-0931-4bc6-b657-9e34096c9dca None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:12.567758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e454555f-0931-4bc6-b657-9e34096c9dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:12.578011 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1369/2715] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:12.582420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a91b4-1634-4877-89c5-a321c7103935 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:12.592991 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a91b4-1634-4877-89c5-a321c7103935 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:12.599084 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0caddc-1b1e-422c-8da9-2054d3a67580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:12.599861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0caddc-1b1e-422c-8da9-2054d3a67580 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:12.600243 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0caddc-1b1e-422c-8da9-2054d3a67580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:12.600863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0caddc-1b1e-422c-8da9-2054d3a67580 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:12.601726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0caddc-1b1e-422c-8da9-2054d3a67580 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:19:12.602177 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0caddc-1b1e-422c-8da9-2054d3a67580 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:12.604159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0caddc-1b1e-422c-8da9-2054d3a67580 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:12.606090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:12.630463 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454555f-0931-4bc6-b657-9e34096c9dca None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:12.637612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1347/2716] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:19:12.642183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454555f-0931-4bc6-b657-9e34096c9dca None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:12.648944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccbcd0-655b-4180-a43e-e33d5d7fe635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:12.649854 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccbcd0-655b-4180-a43e-e33d5d7fe635 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:12.650406 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccbcd0-655b-4180-a43e-e33d5d7fe635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:12.650752 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccbcd0-655b-4180-a43e-e33d5d7fe635 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:12.651810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccbcd0-655b-4180-a43e-e33d5d7fe635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:12.654738 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccbcd0-655b-4180-a43e-e33d5d7fe635 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:12.702865 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1370/2717] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:12 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:12.709221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0caddc-1b1e-422c-8da9-2054d3a67580 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:12.721379 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0caddc-1b1e-422c-8da9-2054d3a67580 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:12.782313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e05cfd-f127-4a37-9468-87ac68e92ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:12.782734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e05cfd-f127-4a37-9468-87ac68e92ade admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:12.783147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e05cfd-f127-4a37-9468-87ac68e92ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:12.783317 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e05cfd-f127-4a37-9468-87ac68e92ade admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:12.783926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e05cfd-f127-4a37-9468-87ac68e92ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:12.785904 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e05cfd-f127-4a37-9468-87ac68e92ade admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:12.817861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1348/2718] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:12 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:12.819396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccbcd0-655b-4180-a43e-e33d5d7fe635 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:12.831687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccbcd0-655b-4180-a43e-e33d5d7fe635 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:12.836875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4949-0e20-4ec9-a6c7-499eab142ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:12.837755 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4949-0e20-4ec9-a6c7-499eab142ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:12.838120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4949-0e20-4ec9-a6c7-499eab142ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:12.838483 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4949-0e20-4ec9-a6c7-499eab142ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:12.839388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd4949-0e20-4ec9-a6c7-499eab142ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:12.841040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd4949-0e20-4ec9-a6c7-499eab142ff3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:12.846342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e05cfd-f127-4a37-9468-87ac68e92ade admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:12.848275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-79e05cfd-f127-4a37-9468-87ac68e92ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:12.856462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1371/2719] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:12.860293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4949-0e20-4ec9-a6c7-499eab142ff3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:12.870875 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4949-0e20-4ec9-a6c7-499eab142ff3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:12.876677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9ed86-d79d-49c0-a910-03a699ad3b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:12.877687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9ed86-d79d-49c0-a910-03a699ad3b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:12.877988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9ed86-d79d-49c0-a910-03a699ad3b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:12.878288 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9ed86-d79d-49c0-a910-03a699ad3b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:12.878887 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9ed86-d79d-49c0-a910-03a699ad3b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:12.880810 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9ed86-d79d-49c0-a910-03a699ad3b35 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:12.895722 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1372/2720] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:12.899672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9ed86-d79d-49c0-a910-03a699ad3b35 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:12.912980 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9ed86-d79d-49c0-a910-03a699ad3b35 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:12.930506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7fb65-c6cb-4028-8f91-64826b0b0580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:12.931347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7fb65-c6cb-4028-8f91-64826b0b0580 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:12.931695 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7fb65-c6cb-4028-8f91-64826b0b0580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:12.932040 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7fb65-c6cb-4028-8f91-64826b0b0580 admin admin] PATH_INFO: `/v3/projects/ca0102d4e54d48bc94d2f2191afffe52/users/a2b4741d6c8d409c9c947d6cd4cdcf1f/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:12.943216 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e05cfd-f127-4a37-9468-87ac68e92ade admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:12.950099 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd2785-6de2-4783-966e-a92fe3084f06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:12.950919 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd2785-6de2-4783-966e-a92fe3084f06 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:12.951383 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd2785-6de2-4783-966e-a92fe3084f06 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:12.951536 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7fb65-c6cb-4028-8f91-64826b0b0580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0102d4e54d48bc94d2f2191afffe52, user_id=a2b4741d6c8d409c9c947d6cd4cdcf1f, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:12.951945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd2785-6de2-4783-966e-a92fe3084f06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:12.955177 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7fb65-c6cb-4028-8f91-64826b0b0580 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:12.962984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-10fd2785-6de2-4783-966e-a92fe3084f06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:12.978176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd2785-6de2-4783-966e-a92fe3084f06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:12.988803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1373/2721] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:12 2022] PUT /identity/v3/projects/ca0102d4e54d48bc94d2f2191afffe52/users/a2b4741d6c8d409c9c947d6cd4cdcf1f/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:12.989020 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd2785-6de2-4783-966e-a92fe3084f06 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:12.990527 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007acff-6e39-4af3-90a8-21849bcafe7a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:12.990866 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007acff-6e39-4af3-90a8-21849bcafe7a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:12.991301 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007acff-6e39-4af3-90a8-21849bcafe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:13.011509 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1349/2722] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:19:13.032458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5007acff-6e39-4af3-90a8-21849bcafe7a None None] MFA Rules not processed for user `a2b4741d6c8d409c9c947d6cd4cdcf1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:13.033680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5007acff-6e39-4af3-90a8-21849bcafe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:13.052908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:13.143827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1374/2723] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:13.148215 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd2785-6de2-4783-966e-a92fe3084f06 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:13.168244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd2785-6de2-4783-966e-a92fe3084f06 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:13.246080 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d56f96-6001-4349-b9bf-a23f2605d037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:13.246984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d56f96-6001-4349-b9bf-a23f2605d037 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:13.247256 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d56f96-6001-4349-b9bf-a23f2605d037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:13.247808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d56f96-6001-4349-b9bf-a23f2605d037 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:13.248489 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d56f96-6001-4349-b9bf-a23f2605d037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:13.251282 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d56f96-6001-4349-b9bf-a23f2605d037 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:13.269267 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:13.289386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1350/2724] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:13 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:13.290605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007acff-6e39-4af3-90a8-21849bcafe7a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:13.307015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007acff-6e39-4af3-90a8-21849bcafe7a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:13.382635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bb00e-a36c-4d5d-adcf-f830453f8139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:13.383709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bb00e-a36c-4d5d-adcf-f830453f8139 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:13.383990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bb00e-a36c-4d5d-adcf-f830453f8139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:13.384264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bb00e-a36c-4d5d-adcf-f830453f8139 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:13.384946 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744bb00e-a36c-4d5d-adcf-f830453f8139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:13.386904 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744bb00e-a36c-4d5d-adcf-f830453f8139 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:13.413178 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1375/2725] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:13 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:13.414722 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d56f96-6001-4349-b9bf-a23f2605d037 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:13.430653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d56f96-6001-4349-b9bf-a23f2605d037 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:13.436796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dceb9-b4f5-43f8-9a49-16a73e739262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:13.437157 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dceb9-b4f5-43f8-9a49-16a73e739262 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:13.437505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dceb9-b4f5-43f8-9a49-16a73e739262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:13.437909 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dceb9-b4f5-43f8-9a49-16a73e739262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:13.439171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dceb9-b4f5-43f8-9a49-16a73e739262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:13.440996 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dceb9-b4f5-43f8-9a49-16a73e739262 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:13.457571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1351/2726] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:13.462329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bb00e-a36c-4d5d-adcf-f830453f8139 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:13.481591 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bb00e-a36c-4d5d-adcf-f830453f8139 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:13.491428 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf561e3d-8bde-42d9-b205-74cec7dfd7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:13.492607 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf561e3d-8bde-42d9-b205-74cec7dfd7be admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:13.493149 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf561e3d-8bde-42d9-b205-74cec7dfd7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:13.494462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf561e3d-8bde-42d9-b205-74cec7dfd7be admin admin] PATH_INFO: `/v3/projects/1a4d6f15e84641109e1e40fcae1b888e/users/b173f9186a974e6890940e48b7b96a07/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:13.516356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf561e3d-8bde-42d9-b205-74cec7dfd7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4d6f15e84641109e1e40fcae1b888e, user_id=b173f9186a974e6890940e48b7b96a07, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:13.521253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf561e3d-8bde-42d9-b205-74cec7dfd7be admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:13.556878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1376/2727] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:13 2022] PUT /identity/v3/projects/1a4d6f15e84641109e1e40fcae1b888e/users/b173f9186a974e6890940e48b7b96a07/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:13.558210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dceb9-b4f5-43f8-9a49-16a73e739262 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:13.589571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dceb9-b4f5-43f8-9a49-16a73e739262 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:13.667195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7c298-60ab-4338-be29-dd561f281178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:13.667922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7c298-60ab-4338-be29-dd561f281178 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:13.668783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7c298-60ab-4338-be29-dd561f281178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:13.668783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7c298-60ab-4338-be29-dd561f281178 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:13.669285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7c298-60ab-4338-be29-dd561f281178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:13.671117 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7c298-60ab-4338-be29-dd561f281178 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:13.684148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1352/2728] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:13.689027 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf561e3d-8bde-42d9-b205-74cec7dfd7be admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:13.700292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf561e3d-8bde-42d9-b205-74cec7dfd7be admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:13.706823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ded62-8d2e-4b7b-b06c-1dda5840ab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:13.707588 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ded62-8d2e-4b7b-b06c-1dda5840ab59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:13.707841 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ded62-8d2e-4b7b-b06c-1dda5840ab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:13.708107 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ded62-8d2e-4b7b-b06c-1dda5840ab59 admin admin] PATH_INFO: `/v3/projects/1a4d6f15e84641109e1e40fcae1b888e/users/b173f9186a974e6890940e48b7b96a07/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:13.711729 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ded62-8d2e-4b7b-b06c-1dda5840ab59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4d6f15e84641109e1e40fcae1b888e, user_id=b173f9186a974e6890940e48b7b96a07, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:13.715458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ded62-8d2e-4b7b-b06c-1dda5840ab59 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:13.727091 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1377/2729] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:13 2022] PUT /identity/v3/projects/1a4d6f15e84641109e1e40fcae1b888e/users/b173f9186a974e6890940e48b7b96a07/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:13.738459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61e263-947a-4f95-96d3-a75f03ca3334 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:13.738729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61e263-947a-4f95-96d3-a75f03ca3334 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:13.739140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61e263-947a-4f95-96d3-a75f03ca3334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:13.776731 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9e61e263-947a-4f95-96d3-a75f03ca3334 None None] MFA Rules not processed for user `b173f9186a974e6890940e48b7b96a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:13.777962 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9e61e263-947a-4f95-96d3-a75f03ca3334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:13.853947 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1353/2730] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:13.866595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ded62-8d2e-4b7b-b06c-1dda5840ab59 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:13.868394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-920ded62-8d2e-4b7b-b06c-1dda5840ab59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:13.930531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ded62-8d2e-4b7b-b06c-1dda5840ab59 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:13.934354 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2232b-1593-4c78-95f1-b087bf6dd266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:13.934968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2232b-1593-4c78-95f1-b087bf6dd266 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:13.935448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2232b-1593-4c78-95f1-b087bf6dd266 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:13.935586 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2232b-1593-4c78-95f1-b087bf6dd266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:13.945631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:13.995595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2232b-1593-4c78-95f1-b087bf6dd266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:13.997037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2232b-1593-4c78-95f1-b087bf6dd266 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:14.008130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1378/2731] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:16.088697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61e263-947a-4f95-96d3-a75f03ca3334 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:16.090197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9e61e263-947a-4f95-96d3-a75f03ca3334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:16.153528 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61e263-947a-4f95-96d3-a75f03ca3334 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:16.157972 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:16.158591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:16.158863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:16.159160 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] PATH_INFO: `/v3/users/d54c900e2947403ebb384282b33385cb` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:16.160252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d54c900e2947403ebb384282b33385cb)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:16.162122 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:16.166564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:16.209847 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd54c900e2947403ebb384282b33385cb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:16.222935 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd54c900e2947403ebb384282b33385cb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:16.229861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd54c900e2947403ebb384282b33385cb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:16.230565 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd54c900e2947403ebb384282b33385cb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:16.238465 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd54c900e2947403ebb384282b33385cb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:16.244319 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1354/2732] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:16 2022] DELETE /identity/v3/users/d54c900e2947403ebb384282b33385cb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:16.248787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f530c-6f89-410a-81d0-d73de8004161 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:16.249322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f530c-6f89-410a-81d0-d73de8004161 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:16.249322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f530c-6f89-410a-81d0-d73de8004161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:16.268976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d27f530c-6f89-410a-81d0-d73de8004161 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:16.270170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d27f530c-6f89-410a-81d0-d73de8004161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:16.327781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1379/2733] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:16.339260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:16.340725 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:16.396706 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c9479-0daa-4f0d-ab73-a4f3dc9ee269 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:16.400337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc757d5-eaf6-4bec-a6b3-f6136f233926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:16.400930 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc757d5-eaf6-4bec-a6b3-f6136f233926 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:16.401203 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc757d5-eaf6-4bec-a6b3-f6136f233926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:16.401470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc757d5-eaf6-4bec-a6b3-f6136f233926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:16.410490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc757d5-eaf6-4bec-a6b3-f6136f233926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:16.411932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc757d5-eaf6-4bec-a6b3-f6136f233926 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:16.421147 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1355/2734] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:19:16.609961 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f530c-6f89-410a-81d0-d73de8004161 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:16.611534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d27f530c-6f89-410a-81d0-d73de8004161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:16.629349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f530c-6f89-410a-81d0-d73de8004161 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:16.634306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:16.635193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:16.635651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:16.636104 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] PATH_INFO: `/v3/users/a2b4741d6c8d409c9c947d6cd4cdcf1f` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:16.637498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b4741d6c8d409c9c947d6cd4cdcf1f)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:16.641472 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:16.704816 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b4741d6c8d409c9c947d6cd4cdcf1f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:16.721429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b4741d6c8d409c9c947d6cd4cdcf1f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:16.731135 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b4741d6c8d409c9c947d6cd4cdcf1f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:16.745334 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b4741d6c8d409c9c947d6cd4cdcf1f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:16.746102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b4741d6c8d409c9c947d6cd4cdcf1f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:16.753768 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1380/2735] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:16 2022] DELETE /identity/v3/users/a2b4741d6c8d409c9c947d6cd4cdcf1f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:16.760878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c24f8b-481d-4685-9cf2-2d050cf911e8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:16.761682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c24f8b-481d-4685-9cf2-2d050cf911e8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:16.762196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c24f8b-481d-4685-9cf2-2d050cf911e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:16.783590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-93c24f8b-481d-4685-9cf2-2d050cf911e8 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:16.784802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-93c24f8b-481d-4685-9cf2-2d050cf911e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:16.853726 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1356/2736] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:16.869445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:16.870873 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:16.899110 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:16.932688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fb32d-7b0f-4a30-ad39-f6f8750a3375 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:16.936336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc0fec-6824-4f15-8608-4de6ebc5165b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:16.936944 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc0fec-6824-4f15-8608-4de6ebc5165b service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:16.937217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc0fec-6824-4f15-8608-4de6ebc5165b service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:16.937492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc0fec-6824-4f15-8608-4de6ebc5165b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:16.946897 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc0fec-6824-4f15-8608-4de6ebc5165b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:16.948332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc0fec-6824-4f15-8608-4de6ebc5165b service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:16.958207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1381/2737] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:19:16.986925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c24f8b-481d-4685-9cf2-2d050cf911e8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:16.989261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-93c24f8b-481d-4685-9cf2-2d050cf911e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:17.019272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c24f8b-481d-4685-9cf2-2d050cf911e8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:17.024951 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:17.025595 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:17.025876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:17.026261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] PATH_INFO: `/v3/projects/9e16a653e64b4c0aa739a877fa73bf76` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:17.027640 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e16a653e64b4c0aa739a877fa73bf76)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:17.029682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:17.055984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:17.083815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e16a653e64b4c0aa739a877fa73bf76', 'request_id': 'req-06a0a27d-4835-4099-8842-a34f6dd7d8ef'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.090398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e16a653e64b4c0aa739a877fa73bf76', 'request_id': 'req-06a0a27d-4835-4099-8842-a34f6dd7d8ef'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.097665 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e16a653e64b4c0aa739a877fa73bf76', 'request_id': 'req-06a0a27d-4835-4099-8842-a34f6dd7d8ef'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.115553 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] The token cache is being invalidate because project 9e16a653e64b4c0aa739a877fa73bf76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:17.115911 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.116570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.118040 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1357/2738] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:16 2022] DELETE /identity/v3/projects/9e16a653e64b4c0aa739a877fa73bf76 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:17.123032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc0fec-6824-4f15-8608-4de6ebc5165b service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:17.134737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc0fec-6824-4f15-8608-4de6ebc5165b service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:17.195513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:17.196184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:17.196616 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:17.196898 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] PATH_INFO: `/v3/users/42b1791bea284110bce277f43e11d0e1` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:17.198056 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b1791bea284110bce277f43e11d0e1)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:17.199997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:17.248261 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:17.249792 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:17.253235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b1791bea284110bce277f43e11d0e1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.265027 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0a27d-4835-4099-8842-a34f6dd7d8ef admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:17.268602 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b1791bea284110bce277f43e11d0e1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.272295 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:17.272977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:17.273323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:17.273601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] PATH_INFO: `/v3/projects/ca0102d4e54d48bc94d2f2191afffe52` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:17.275241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca0102d4e54d48bc94d2f2191afffe52)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:17.278227 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:17.279538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b1791bea284110bce277f43e11d0e1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.289346 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b1791bea284110bce277f43e11d0e1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.289979 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b1791bea284110bce277f43e11d0e1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.296374 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1382/2739] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:17 2022] DELETE /identity/v3/users/42b1791bea284110bce277f43e11d0e1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:17.332717 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca0102d4e54d48bc94d2f2191afffe52', 'request_id': 'req-d8be3866-6677-4906-ad41-369ca3c2cd2d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.338861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca0102d4e54d48bc94d2f2191afffe52', 'request_id': 'req-d8be3866-6677-4906-ad41-369ca3c2cd2d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.345880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca0102d4e54d48bc94d2f2191afffe52', 'request_id': 'req-d8be3866-6677-4906-ad41-369ca3c2cd2d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.359705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] The token cache is being invalidate because project ca0102d4e54d48bc94d2f2191afffe52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:17.359998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.360736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.362570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1358/2740] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:17 2022] DELETE /identity/v3/projects/ca0102d4e54d48bc94d2f2191afffe52 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:17.366492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:17.368429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:17.421876 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:17.439207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5452e6-fa53-4847-be3f-82fcb9952bfe admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:17.504477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:17.505150 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:17.505419 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:17.505693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] PATH_INFO: `/v3/users/b173f9186a974e6890940e48b7b96a07` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:17.506839 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b173f9186a974e6890940e48b7b96a07)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:17.508775 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:17.563345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b173f9186a974e6890940e48b7b96a07'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.579819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b173f9186a974e6890940e48b7b96a07'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.582927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:17.584590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:17.590475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b173f9186a974e6890940e48b7b96a07'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.599313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be3866-6677-4906-ad41-369ca3c2cd2d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:17.603803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b173f9186a974e6890940e48b7b96a07'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.604477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b173f9186a974e6890940e48b7b96a07'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.609666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:17.610328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:17.610609 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:17.610977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] PATH_INFO: `/v3/projects/bcb2e6ec21004824b3b935421a89e77d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:17.612158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb2e6ec21004824b3b935421a89e77d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:17.614065 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:17.616477 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1383/2741] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:17 2022] DELETE /identity/v3/users/b173f9186a974e6890940e48b7b96a07 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:17.676312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb2e6ec21004824b3b935421a89e77d', 'request_id': 'req-f63e02dc-8921-468a-8cee-13a8b49bf4ea'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.682705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb2e6ec21004824b3b935421a89e77d', 'request_id': 'req-f63e02dc-8921-468a-8cee-13a8b49bf4ea'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.690488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb2e6ec21004824b3b935421a89e77d', 'request_id': 'req-f63e02dc-8921-468a-8cee-13a8b49bf4ea'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.704543 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] The token cache is being invalidate because project bcb2e6ec21004824b3b935421a89e77d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:17.704834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.705485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.707007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1359/2742] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:17 2022] DELETE /identity/v3/projects/bcb2e6ec21004824b3b935421a89e77d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:17.711190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:17.712730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:17.805581 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58510321-103a-4eeb-81c0-b5c00adb80f4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:17.827920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:17.883963 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:17.890630 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:17.890941 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:17.891247 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] PATH_INFO: `/v3/users/80265799eb704a4fb6728e68506f022c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:17.892545 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] RBAC: Authorizing `identity:delete_user(user_id=80265799eb704a4fb6728e68506f022c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:17.894478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:17.901173 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:17.903158 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:17.928057 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=21,keystone:SELECT=76,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:17.957348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80265799eb704a4fb6728e68506f022c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.978958 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80265799eb704a4fb6728e68506f022c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.981878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e02dc-8921-468a-8cee-13a8b49bf4ea admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:17.986909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80265799eb704a4fb6728e68506f022c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.988245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab52dd2-42d4-4eb7-ab5a-3f8a49c883c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:17.989049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab52dd2-42d4-4eb7-ab5a-3f8a49c883c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:17.989325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab52dd2-42d4-4eb7-ab5a-3f8a49c883c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:17.989774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab52dd2-42d4-4eb7-ab5a-3f8a49c883c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:17.991653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-dab52dd2-42d4-4eb7-ab5a-3f8a49c883c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:17.997320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80265799eb704a4fb6728e68506f022c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:17.997995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80265799eb704a4fb6728e68506f022c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:18.005396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1384/2743] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:17 2022] DELETE /identity/v3/users/80265799eb704a4fb6728e68506f022c => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:18.006257 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:18.009151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab52dd2-42d4-4eb7-ab5a-3f8a49c883c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:18.010604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab52dd2-42d4-4eb7-ab5a-3f8a49c883c5 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:18.011324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:18.022252 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1360/2744] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:18.080912 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff633329-1cb0-4a85-b288-30589068987d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:18.085746 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:18.086389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.086680 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.087084 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] PATH_INFO: `/v3/projects/1a4d6f15e84641109e1e40fcae1b888e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.088146 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a4d6f15e84641109e1e40fcae1b888e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:18.090062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:18.150933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:18.158916 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a4d6f15e84641109e1e40fcae1b888e', 'request_id': 'req-47916fbd-cc7d-4152-9826-ac9608611d1a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:18.166721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a4d6f15e84641109e1e40fcae1b888e', 'request_id': 'req-47916fbd-cc7d-4152-9826-ac9608611d1a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:18.174126 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a4d6f15e84641109e1e40fcae1b888e', 'request_id': 'req-47916fbd-cc7d-4152-9826-ac9608611d1a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:18.190356 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] The token cache is being invalidate because project 1a4d6f15e84641109e1e40fcae1b888e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:18.190674 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:18.191400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-47916fbd-cc7d-4152-9826-ac9608611d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:18.193030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1385/2745] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:18 2022] DELETE /identity/v3/projects/1a4d6f15e84641109e1e40fcae1b888e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:18.202010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59877ab6-b201-41ea-ba3a-059d9d5f5f20 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.202657 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59877ab6-b201-41ea-ba3a-059d9d5f5f20 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.203272 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59877ab6-b201-41ea-ba3a-059d9d5f5f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.229981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-59877ab6-b201-41ea-ba3a-059d9d5f5f20 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:18.231260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-59877ab6-b201-41ea-ba3a-059d9d5f5f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:18.301526 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1361/2746] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:18.311821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700a697-1e11-4fb6-a341-90c46252af19 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.312678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700a697-1e11-4fb6-a341-90c46252af19 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.313079 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700a697-1e11-4fb6-a341-90c46252af19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.335267 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6700a697-1e11-4fb6-a341-90c46252af19 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:18.336655 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6700a697-1e11-4fb6-a341-90c46252af19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:18.346212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1386/2747] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:18.351106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59877ab6-b201-41ea-ba3a-059d9d5f5f20 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:18.363988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59877ab6-b201-41ea-ba3a-059d9d5f5f20 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:18.372226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78daa65-caa6-4799-b9f8-521ae362e14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:18.372861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78daa65-caa6-4799-b9f8-521ae362e14d admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.373209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78daa65-caa6-4799-b9f8-521ae362e14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.373546 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78daa65-caa6-4799-b9f8-521ae362e14d admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.374270 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78daa65-caa6-4799-b9f8-521ae362e14d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:19:18.374637 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78daa65-caa6-4799-b9f8-521ae362e14d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:18.376426 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78daa65-caa6-4799-b9f8-521ae362e14d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:18.390820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1362/2748] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:19:18.396679 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700a697-1e11-4fb6-a341-90c46252af19 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:18.410638 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700a697-1e11-4fb6-a341-90c46252af19 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:18.417089 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de961a-f85d-4110-8ba0-4a6ebaaf1256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:18.418176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de961a-f85d-4110-8ba0-4a6ebaaf1256 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.418554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de961a-f85d-4110-8ba0-4a6ebaaf1256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.418925 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de961a-f85d-4110-8ba0-4a6ebaaf1256 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.419718 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de961a-f85d-4110-8ba0-4a6ebaaf1256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:18.423145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de961a-f85d-4110-8ba0-4a6ebaaf1256 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:18.464475 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1387/2749] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:18 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:18.469496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78daa65-caa6-4799-b9f8-521ae362e14d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:18.481505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78daa65-caa6-4799-b9f8-521ae362e14d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:18.544908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de961a-f85d-4110-8ba0-4a6ebaaf1256 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:18.548284 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-36de961a-f85d-4110-8ba0-4a6ebaaf1256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:18.563861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283fffe-e760-4b6b-8d90-01907a05d3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:18.565071 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283fffe-e760-4b6b-8d90-01907a05d3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.565482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283fffe-e760-4b6b-8d90-01907a05d3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.565957 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283fffe-e760-4b6b-8d90-01907a05d3c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.566761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283fffe-e760-4b6b-8d90-01907a05d3c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:18.568786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283fffe-e760-4b6b-8d90-01907a05d3c1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:18.590304 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de961a-f85d-4110-8ba0-4a6ebaaf1256 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:18.598719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1363/2750] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:18 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:18.598927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:18.598927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.598927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.598927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] PATH_INFO: `/v3/projects/99b7adc3f8324d1c9e673456643ca4e8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.600115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b7adc3f8324d1c9e673456643ca4e8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:18.600703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283fffe-e760-4b6b-8d90-01907a05d3c1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:18.602726 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:18.612649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283fffe-e760-4b6b-8d90-01907a05d3c1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:18.618770 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcca68a-8fa9-4e3d-aeef-39ce2bd206e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:18.619768 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcca68a-8fa9-4e3d-aeef-39ce2bd206e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.619965 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcca68a-8fa9-4e3d-aeef-39ce2bd206e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.620340 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcca68a-8fa9-4e3d-aeef-39ce2bd206e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.621086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcca68a-8fa9-4e3d-aeef-39ce2bd206e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:18.622787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcca68a-8fa9-4e3d-aeef-39ce2bd206e8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:18.637260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1364/2751] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:18.641151 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcca68a-8fa9-4e3d-aeef-39ce2bd206e8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:18.653611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcca68a-8fa9-4e3d-aeef-39ce2bd206e8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:18.662367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b7adc3f8324d1c9e673456643ca4e8', 'request_id': 'req-0e31f474-b291-4ce9-9752-758b184c0efd'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:18.670369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b7adc3f8324d1c9e673456643ca4e8', 'request_id': 'req-0e31f474-b291-4ce9-9752-758b184c0efd'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:18.678278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b7adc3f8324d1c9e673456643ca4e8', 'request_id': 'req-0e31f474-b291-4ce9-9752-758b184c0efd'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:18.681083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:18.695710 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] The token cache is being invalidate because project 99b7adc3f8324d1c9e673456643ca4e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:18.696171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:18.697787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e31f474-b291-4ce9-9752-758b184c0efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:18.699624 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1388/2752] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:18 2022] DELETE /identity/v3/projects/99b7adc3f8324d1c9e673456643ca4e8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:18.704917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748f18f-bb85-4356-84c4-1ef01e59438c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.705202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748f18f-bb85-4356-84c4-1ef01e59438c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.705481 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748f18f-bb85-4356-84c4-1ef01e59438c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.728871 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f748f18f-bb85-4356-84c4-1ef01e59438c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:18.730351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f748f18f-bb85-4356-84c4-1ef01e59438c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:18.732578 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369512d-7259-43ca-8c8c-ccc84e04eda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:18.733305 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369512d-7259-43ca-8c8c-ccc84e04eda8 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.733686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369512d-7259-43ca-8c8c-ccc84e04eda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.734082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369512d-7259-43ca-8c8c-ccc84e04eda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.735451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369512d-7259-43ca-8c8c-ccc84e04eda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:18.737131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369512d-7259-43ca-8c8c-ccc84e04eda8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:18.743976 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1389/2753] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:18.750998 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1365/2754] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:18.753432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92f457-4728-4c05-8df3-75dd7a49c3da None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.753783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92f457-4728-4c05-8df3-75dd7a49c3da None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.754128 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92f457-4728-4c05-8df3-75dd7a49c3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.754421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369512d-7259-43ca-8c8c-ccc84e04eda8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:18.758563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a369512d-7259-43ca-8c8c-ccc84e04eda8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:18.772914 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369512d-7259-43ca-8c8c-ccc84e04eda8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:18.778035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ee0db-9a0f-42b6-8fa2-379e6215318e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:18.778694 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ee0db-9a0f-42b6-8fa2-379e6215318e admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.778826 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8e92f457-4728-4c05-8df3-75dd7a49c3da None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:18.779602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ee0db-9a0f-42b6-8fa2-379e6215318e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.779602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ee0db-9a0f-42b6-8fa2-379e6215318e admin admin] PATH_INFO: `/v3/projects/1d274d2bacf045dc97c1c92ce862a90d/users/377ece01a4284b7aa3db6d58c1733c4c/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.780203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8e92f457-4728-4c05-8df3-75dd7a49c3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:18.794422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1390/2755] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:18.796114 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ee0db-9a0f-42b6-8fa2-379e6215318e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d274d2bacf045dc97c1c92ce862a90d, user_id=377ece01a4284b7aa3db6d58c1733c4c, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:18.799037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92f457-4728-4c05-8df3-75dd7a49c3da None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:18.799411 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ee0db-9a0f-42b6-8fa2-379e6215318e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:18.809074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92f457-4728-4c05-8df3-75dd7a49c3da None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:18.816225 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1366/2756] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:18 2022] PUT /identity/v3/projects/1d274d2bacf045dc97c1c92ce862a90d/users/377ece01a4284b7aa3db6d58c1733c4c/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:18.827448 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732bb93-d643-45df-9851-dd1e1b356e9a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.827739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732bb93-d643-45df-9851-dd1e1b356e9a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.829115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732bb93-d643-45df-9851-dd1e1b356e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.859900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e732bb93-d643-45df-9851-dd1e1b356e9a None None] MFA Rules not processed for user `377ece01a4284b7aa3db6d58c1733c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:18.861244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e732bb93-d643-45df-9851-dd1e1b356e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:18.873324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8763b-0243-44cc-a9b5-743df66fc42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:18.873882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8763b-0243-44cc-a9b5-743df66fc42f admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.874278 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8763b-0243-44cc-a9b5-743df66fc42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.874441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8763b-0243-44cc-a9b5-743df66fc42f admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.875130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8763b-0243-44cc-a9b5-743df66fc42f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:19:18.875418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8763b-0243-44cc-a9b5-743df66fc42f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:18.877029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8763b-0243-44cc-a9b5-743df66fc42f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:18.889353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1391/2757] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 17:19:18.894366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8763b-0243-44cc-a9b5-743df66fc42f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:18.900962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:18.905987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8763b-0243-44cc-a9b5-743df66fc42f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:18.913235 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32bb9c-6f1b-405e-b472-d79fac11c99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:18.914145 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32bb9c-6f1b-405e-b472-d79fac11c99c admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:18.914512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32bb9c-6f1b-405e-b472-d79fac11c99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:18.914776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32bb9c-6f1b-405e-b472-d79fac11c99c admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:18.915836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32bb9c-6f1b-405e-b472-d79fac11c99c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:18.917954 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32bb9c-6f1b-405e-b472-d79fac11c99c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:18.941159 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1367/2758] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:19:18.955448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1392/2759] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:18 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:19:18.955649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732bb93-d643-45df-9851-dd1e1b356e9a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:18.955649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e732bb93-d643-45df-9851-dd1e1b356e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:18.957270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32bb9c-6f1b-405e-b472-d79fac11c99c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:19.018405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732bb93-d643-45df-9851-dd1e1b356e9a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:19.027855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32bb9c-6f1b-405e-b472-d79fac11c99c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:19.035216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7801688-a804-4778-847f-c25e4e6336df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:19.036371 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7801688-a804-4778-847f-c25e4e6336df admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:19.036709 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7801688-a804-4778-847f-c25e4e6336df admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:19.037136 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7801688-a804-4778-847f-c25e4e6336df admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:19.037787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7801688-a804-4778-847f-c25e4e6336df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:19.039867 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7801688-a804-4778-847f-c25e4e6336df admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:19.072764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1393/2760] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:18 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:19.076824 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7801688-a804-4778-847f-c25e4e6336df admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:19.079061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:19.092461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7801688-a804-4778-847f-c25e4e6336df admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:19.095868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cf766-516c-46b9-96c2-669b64e74844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:19.096673 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cf766-516c-46b9-96c2-669b64e74844 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:19.097152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cf766-516c-46b9-96c2-669b64e74844 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:19.097591 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cf766-516c-46b9-96c2-669b64e74844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:19.100671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e795822-13bf-4c08-b04e-7fbf7ab336a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:19.101363 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e795822-13bf-4c08-b04e-7fbf7ab336a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:19.101704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e795822-13bf-4c08-b04e-7fbf7ab336a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:19.102120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e795822-13bf-4c08-b04e-7fbf7ab336a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:19.102836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e795822-13bf-4c08-b04e-7fbf7ab336a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:19.105531 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e795822-13bf-4c08-b04e-7fbf7ab336a2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:19.121379 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1394/2761] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:19.125919 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e795822-13bf-4c08-b04e-7fbf7ab336a2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:19.137702 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e795822-13bf-4c08-b04e-7fbf7ab336a2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:19.144463 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9171e-7503-48d2-b998-65fd3e6df5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:19.145246 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9171e-7503-48d2-b998-65fd3e6df5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:19.145528 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9171e-7503-48d2-b998-65fd3e6df5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:19.145800 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9171e-7503-48d2-b998-65fd3e6df5a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:19.146583 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9171e-7503-48d2-b998-65fd3e6df5a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:19.148184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9171e-7503-48d2-b998-65fd3e6df5a3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:19.164672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1395/2762] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:19.166044 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9171e-7503-48d2-b998-65fd3e6df5a3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:19.178858 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9171e-7503-48d2-b998-65fd3e6df5a3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:19.183950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9cf766-516c-46b9-96c2-669b64e74844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:19.185204 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac2f12-e6a0-47f7-82f2-5b37bca59ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:19.185917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac2f12-e6a0-47f7-82f2-5b37bca59ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:19.186086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9cf766-516c-46b9-96c2-669b64e74844 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:19.186217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac2f12-e6a0-47f7-82f2-5b37bca59ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:19.186446 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac2f12-e6a0-47f7-82f2-5b37bca59ab0 admin admin] PATH_INFO: `/v3/projects/237f3a24bb3a474ca9b504e55905108d/users/ac2d30454a654abe92d7b1fbeb82e167/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:19.198325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1368/2763] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:19:19.205015 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac2f12-e6a0-47f7-82f2-5b37bca59ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=237f3a24bb3a474ca9b504e55905108d, user_id=ac2d30454a654abe92d7b1fbeb82e167, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:19.208114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac2f12-e6a0-47f7-82f2-5b37bca59ab0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:19.224971 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1396/2764] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:19 2022] PUT /identity/v3/projects/237f3a24bb3a474ca9b504e55905108d/users/ac2d30454a654abe92d7b1fbeb82e167/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:19.238525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f24bf-a6da-4fe9-9a6e-ee5c21b5b988 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:19.239042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f24bf-a6da-4fe9-9a6e-ee5c21b5b988 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:19.239245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f24bf-a6da-4fe9-9a6e-ee5c21b5b988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:19.271958 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d13f24bf-a6da-4fe9-9a6e-ee5c21b5b988 None None] MFA Rules not processed for user `ac2d30454a654abe92d7b1fbeb82e167`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:19.273287 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d13f24bf-a6da-4fe9-9a6e-ee5c21b5b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:19.361396 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1369/2765] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:19.379078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac2f12-e6a0-47f7-82f2-5b37bca59ab0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:19.384006 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e1ac2f12-e6a0-47f7-82f2-5b37bca59ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:19.455465 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac2f12-e6a0-47f7-82f2-5b37bca59ab0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:19.459393 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66360d3-4dfc-464b-938e-2e590df7fa78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:19.460168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66360d3-4dfc-464b-938e-2e590df7fa78 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:19.460538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66360d3-4dfc-464b-938e-2e590df7fa78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:19.460909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66360d3-4dfc-464b-938e-2e590df7fa78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:19.472330 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66360d3-4dfc-464b-938e-2e590df7fa78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:19.474464 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66360d3-4dfc-464b-938e-2e590df7fa78 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:19.486318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1397/2766] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:21.542165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f24bf-a6da-4fe9-9a6e-ee5c21b5b988 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:21.543763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d13f24bf-a6da-4fe9-9a6e-ee5c21b5b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:21.620395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f24bf-a6da-4fe9-9a6e-ee5c21b5b988 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:21.624854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef4e93-ce2d-4e44-b2d1-862b555d033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:21.626110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef4e93-ce2d-4e44-b2d1-862b555d033e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:21.626363 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef4e93-ce2d-4e44-b2d1-862b555d033e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:21.626628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef4e93-ce2d-4e44-b2d1-862b555d033e admin admin] PATH_INFO: `/v3/users/b34e9318826c49ed891dde1f7782898b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:21.628107 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef4e93-ce2d-4e44-b2d1-862b555d033e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b34e9318826c49ed891dde1f7782898b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:21.630138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef4e93-ce2d-4e44-b2d1-862b555d033e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:21.658597 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:21.677586 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8eef4e93-ce2d-4e44-b2d1-862b555d033e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b34e9318826c49ed891dde1f7782898b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:21.689865 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8eef4e93-ce2d-4e44-b2d1-862b555d033e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b34e9318826c49ed891dde1f7782898b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:21.696592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8eef4e93-ce2d-4e44-b2d1-862b555d033e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b34e9318826c49ed891dde1f7782898b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:21.697267 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8eef4e93-ce2d-4e44-b2d1-862b555d033e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b34e9318826c49ed891dde1f7782898b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:21.704801 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-8eef4e93-ce2d-4e44-b2d1-862b555d033e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b34e9318826c49ed891dde1f7782898b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:21.711076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1370/2767] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:21 2022] DELETE /identity/v3/users/b34e9318826c49ed891dde1f7782898b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:21.893435 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66360d3-4dfc-464b-938e-2e590df7fa78 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:21.911817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66360d3-4dfc-464b-938e-2e590df7fa78 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:21.941552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:21.975141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:21.975797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:21.976075 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:21.976347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] PATH_INFO: `/v3/projects/45847b1fd67c4caaa03a36895cd2263a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:21.977549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=45847b1fd67c4caaa03a36895cd2263a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:21.979543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:22.027169 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45847b1fd67c4caaa03a36895cd2263a', 'request_id': 'req-8e08b895-9d68-406e-addc-60484812daaa'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:22.033308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45847b1fd67c4caaa03a36895cd2263a', 'request_id': 'req-8e08b895-9d68-406e-addc-60484812daaa'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:22.038878 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45847b1fd67c4caaa03a36895cd2263a', 'request_id': 'req-8e08b895-9d68-406e-addc-60484812daaa'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:22.051520 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] The token cache is being invalidate because project 45847b1fd67c4caaa03a36895cd2263a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:22.051811 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:22.052485 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-8e08b895-9d68-406e-addc-60484812daaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:22.053963 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1398/2768] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:21 2022] DELETE /identity/v3/projects/45847b1fd67c4caaa03a36895cd2263a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:22.059035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8baea8-eefb-43dd-9fd4-162ca0e66101 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.059320 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8baea8-eefb-43dd-9fd4-162ca0e66101 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.059590 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8baea8-eefb-43dd-9fd4-162ca0e66101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.079894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2d8baea8-eefb-43dd-9fd4-162ca0e66101 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:22.081076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2d8baea8-eefb-43dd-9fd4-162ca0e66101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:22.138180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1371/2769] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:22.146999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c56c5-cde5-4587-9b19-eb93cd8a5099 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.147295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c56c5-cde5-4587-9b19-eb93cd8a5099 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.147568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c56c5-cde5-4587-9b19-eb93cd8a5099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.166809 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2f9c56c5-cde5-4587-9b19-eb93cd8a5099 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:22.168038 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2f9c56c5-cde5-4587-9b19-eb93cd8a5099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:22.176106 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1399/2770] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:22.180571 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8baea8-eefb-43dd-9fd4-162ca0e66101 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:22.192416 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8baea8-eefb-43dd-9fd4-162ca0e66101 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:22.198560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348fd89-4136-4656-b196-4a6f4c3dc494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:22.199195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348fd89-4136-4656-b196-4a6f4c3dc494 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.199552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348fd89-4136-4656-b196-4a6f4c3dc494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.199745 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348fd89-4136-4656-b196-4a6f4c3dc494 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.200398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348fd89-4136-4656-b196-4a6f4c3dc494 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:19:22.200692 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348fd89-4136-4656-b196-4a6f4c3dc494 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:22.202347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348fd89-4136-4656-b196-4a6f4c3dc494 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:22.215807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1372/2771] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:19:22.227266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c56c5-cde5-4587-9b19-eb93cd8a5099 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:22.229950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348fd89-4136-4656-b196-4a6f4c3dc494 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:22.231915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c348fd89-4136-4656-b196-4a6f4c3dc494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:22.239521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c56c5-cde5-4587-9b19-eb93cd8a5099 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:22.246352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348fd89-4136-4656-b196-4a6f4c3dc494 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:22.251817 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc6caf-d8ab-488a-900e-7df1443e251c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:22.252475 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc6caf-d8ab-488a-900e-7df1443e251c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.252756 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc6caf-d8ab-488a-900e-7df1443e251c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.253028 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc6caf-d8ab-488a-900e-7df1443e251c admin admin] PATH_INFO: `/v3/users/ac2d30454a654abe92d7b1fbeb82e167` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.254176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc6caf-d8ab-488a-900e-7df1443e251c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2d30454a654abe92d7b1fbeb82e167)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:22.256141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc6caf-d8ab-488a-900e-7df1443e251c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:22.258274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c1fc1-2c3d-405a-8842-fe7b6a860b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:22.259375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c1fc1-2c3d-405a-8842-fe7b6a860b1f admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.259657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c1fc1-2c3d-405a-8842-fe7b6a860b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.259927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c1fc1-2c3d-405a-8842-fe7b6a860b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.260570 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c1fc1-2c3d-405a-8842-fe7b6a860b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:22.262447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c1fc1-2c3d-405a-8842-fe7b6a860b1f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:22.317881 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1400/2772] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:22 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:22.319441 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c1fc1-2c3d-405a-8842-fe7b6a860b1f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:22.326659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-07bc6caf-d8ab-488a-900e-7df1443e251c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2d30454a654abe92d7b1fbeb82e167'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:22.343450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-07bc6caf-d8ab-488a-900e-7df1443e251c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2d30454a654abe92d7b1fbeb82e167'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:22.353283 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-07bc6caf-d8ab-488a-900e-7df1443e251c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2d30454a654abe92d7b1fbeb82e167'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:22.354348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-07bc6caf-d8ab-488a-900e-7df1443e251c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2d30454a654abe92d7b1fbeb82e167'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:22.365819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-07bc6caf-d8ab-488a-900e-7df1443e251c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2d30454a654abe92d7b1fbeb82e167'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:22.378973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1373/2773] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:19:22 2022] DELETE /identity/v3/users/ac2d30454a654abe92d7b1fbeb82e167 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:22.385037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e394d5d-3ec7-4c2a-9587-49fbc27e7668 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.385513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e394d5d-3ec7-4c2a-9587-49fbc27e7668 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.386251 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e394d5d-3ec7-4c2a-9587-49fbc27e7668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.394870 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:22.420367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4e394d5d-3ec7-4c2a-9587-49fbc27e7668 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:22.421709 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4e394d5d-3ec7-4c2a-9587-49fbc27e7668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:22.422507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c1fc1-2c3d-405a-8842-fe7b6a860b1f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:22.469776 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:22.500500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1374/2774] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:22.503851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb0d0d-ae60-4c55-85d8-e6ef02829bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:22.503851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb0d0d-ae60-4c55-85d8-e6ef02829bfb admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.503851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb0d0d-ae60-4c55-85d8-e6ef02829bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.503851 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb0d0d-ae60-4c55-85d8-e6ef02829bfb admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.504269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb0d0d-ae60-4c55-85d8-e6ef02829bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:22.508463 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb0d0d-ae60-4c55-85d8-e6ef02829bfb admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:22.519299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e394d5d-3ec7-4c2a-9587-49fbc27e7668 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:22.519299 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4e394d5d-3ec7-4c2a-9587-49fbc27e7668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:22.533928 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1401/2775] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:22 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:22.538423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb0d0d-ae60-4c55-85d8-e6ef02829bfb admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:22.540581 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-20eb0d0d-ae60-4c55-85d8-e6ef02829bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:22.552821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb0d0d-ae60-4c55-85d8-e6ef02829bfb admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:22.557541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8eddf-fb13-4141-9194-715343547eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:22.558099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8eddf-fb13-4141-9194-715343547eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.558378 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8eddf-fb13-4141-9194-715343547eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.558640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8eddf-fb13-4141-9194-715343547eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.559332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8eddf-fb13-4141-9194-715343547eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:22.560911 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8eddf-fb13-4141-9194-715343547eb2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:22.572295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1402/2776] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:19:22.576774 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8eddf-fb13-4141-9194-715343547eb2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:22.582089 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e394d5d-3ec7-4c2a-9587-49fbc27e7668 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:22.585829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df882e0-0019-4263-8e76-591ac52131f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:22.586437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df882e0-0019-4263-8e76-591ac52131f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.586739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df882e0-0019-4263-8e76-591ac52131f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.587051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df882e0-0019-4263-8e76-591ac52131f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.587245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8eddf-fb13-4141-9194-715343547eb2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:22.591966 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca2d0d-d3d9-4488-913c-35c968f6e51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:22.592615 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca2d0d-d3d9-4488-913c-35c968f6e51a admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.592936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca2d0d-d3d9-4488-913c-35c968f6e51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.593213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca2d0d-d3d9-4488-913c-35c968f6e51a admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.594443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca2d0d-d3d9-4488-913c-35c968f6e51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:22.596413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca2d0d-d3d9-4488-913c-35c968f6e51a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:22.597165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df882e0-0019-4263-8e76-591ac52131f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:22.598598 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df882e0-0019-4263-8e76-591ac52131f2 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:22.607220 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1403/2777] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:22.608127 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1375/2778] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:22.611395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca2d0d-d3d9-4488-913c-35c968f6e51a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:22.621174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca2d0d-d3d9-4488-913c-35c968f6e51a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:22.625869 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042deb3-6a4d-4387-b512-76493df8de6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:22.626513 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042deb3-6a4d-4387-b512-76493df8de6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.626795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042deb3-6a4d-4387-b512-76493df8de6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.627109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042deb3-6a4d-4387-b512-76493df8de6f admin admin] PATH_INFO: `/v3/projects/3d75fcb58d0c45f7b7b1dd70e979858f/users/e710fb55566948d3b7091322fd25f3da/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.643078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042deb3-6a4d-4387-b512-76493df8de6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d75fcb58d0c45f7b7b1dd70e979858f, user_id=e710fb55566948d3b7091322fd25f3da, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:22.646027 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042deb3-6a4d-4387-b512-76493df8de6f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:22.661835 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1404/2779] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:22 2022] PUT /identity/v3/projects/3d75fcb58d0c45f7b7b1dd70e979858f/users/e710fb55566948d3b7091322fd25f3da/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:22.673990 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba48a3c-086a-4aec-bc5b-cb21ef5b0f51 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.674356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba48a3c-086a-4aec-bc5b-cb21ef5b0f51 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.674735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba48a3c-086a-4aec-bc5b-cb21ef5b0f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.705679 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3ba48a3c-086a-4aec-bc5b-cb21ef5b0f51 None None] MFA Rules not processed for user `e710fb55566948d3b7091322fd25f3da`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:22.706901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3ba48a3c-086a-4aec-bc5b-cb21ef5b0f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:22.789971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1376/2780] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:22.794456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042deb3-6a4d-4387-b512-76493df8de6f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:22.862787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042deb3-6a4d-4387-b512-76493df8de6f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:22.867512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603291c-ccb4-4fdc-aabe-b64508ab2add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:22.868123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603291c-ccb4-4fdc-aabe-b64508ab2add admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.868403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603291c-ccb4-4fdc-aabe-b64508ab2add admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.868675 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603291c-ccb4-4fdc-aabe-b64508ab2add admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.869341 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603291c-ccb4-4fdc-aabe-b64508ab2add admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:22.871245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603291c-ccb4-4fdc-aabe-b64508ab2add admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:22.901452 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba48a3c-086a-4aec-bc5b-cb21ef5b0f51 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:22.906592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1405/2781] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:22 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:19:22.907133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3ba48a3c-086a-4aec-bc5b-cb21ef5b0f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:22.908322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603291c-ccb4-4fdc-aabe-b64508ab2add admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:22.922027 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba48a3c-086a-4aec-bc5b-cb21ef5b0f51 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:22.962744 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:22.987488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603291c-ccb4-4fdc-aabe-b64508ab2add admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:22.988068 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:22.988789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.989131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.989463 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] PATH_INFO: `/v3/projects/237f3a24bb3a474ca9b504e55905108d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.990650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=237f3a24bb3a474ca9b504e55905108d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:22.992133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61264a54-2baf-4850-8fd2-66ecb07331f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:22.992734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61264a54-2baf-4850-8fd2-66ecb07331f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:22.992850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:22.993016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61264a54-2baf-4850-8fd2-66ecb07331f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:22.993293 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61264a54-2baf-4850-8fd2-66ecb07331f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:22.993920 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61264a54-2baf-4850-8fd2-66ecb07331f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:22.995846 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61264a54-2baf-4850-8fd2-66ecb07331f4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.008404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:23.020141 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1406/2782] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:22 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:19:23.023901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61264a54-2baf-4850-8fd2-66ecb07331f4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.035424 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61264a54-2baf-4850-8fd2-66ecb07331f4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.046329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '237f3a24bb3a474ca9b504e55905108d', 'request_id': 'req-99b9a119-1644-474e-ac94-ac9649260fd4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:23.052485 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '237f3a24bb3a474ca9b504e55905108d', 'request_id': 'req-99b9a119-1644-474e-ac94-ac9649260fd4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:23.059014 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '237f3a24bb3a474ca9b504e55905108d', 'request_id': 'req-99b9a119-1644-474e-ac94-ac9649260fd4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:23.073616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] The token cache is being invalidate because project 237f3a24bb3a474ca9b504e55905108d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:19:23.073897 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:23.075161 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-99b9a119-1644-474e-ac94-ac9649260fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:19:23.076660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1377/2783] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:19:22 2022] DELETE /identity/v3/projects/237f3a24bb3a474ca9b504e55905108d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:23.081887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8890c1-b97f-44d8-bdfe-f699e26b2d0d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.082166 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8890c1-b97f-44d8-bdfe-f699e26b2d0d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.082441 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8890c1-b97f-44d8-bdfe-f699e26b2d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.097786 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ac7e-5cbe-4444-8f44-9c14174884bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.098404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ac7e-5cbe-4444-8f44-9c14174884bd admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.098685 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ac7e-5cbe-4444-8f44-9c14174884bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.099039 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ac7e-5cbe-4444-8f44-9c14174884bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.099687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54ac7e-5cbe-4444-8f44-9c14174884bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.101279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54ac7e-5cbe-4444-8f44-9c14174884bd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.102141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7b8890c1-b97f-44d8-bdfe-f699e26b2d0d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:23.103375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7b8890c1-b97f-44d8-bdfe-f699e26b2d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:23.111735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1378/2784] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:19:23.111917 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1407/2785] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:23.116276 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8890c1-b97f-44d8-bdfe-f699e26b2d0d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.117659 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7b8890c1-b97f-44d8-bdfe-f699e26b2d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:23.119451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e5abb-faf6-452d-84ee-fe6d7cb377df None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.119739 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e5abb-faf6-452d-84ee-fe6d7cb377df None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.120095 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e5abb-faf6-452d-84ee-fe6d7cb377df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.130037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8890c1-b97f-44d8-bdfe-f699e26b2d0d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.135059 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354a468-4c4e-42f1-b684-54be12611e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.135696 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354a468-4c4e-42f1-b684-54be12611e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.135973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354a468-4c4e-42f1-b684-54be12611e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.136242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354a468-4c4e-42f1-b684-54be12611e86 admin admin] PATH_INFO: `/v3/projects/2a8ac27527654484b8e201d7b47fffd9/users/885b6e6f9613498baf0e071d32acd095/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.140482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ab6e5abb-faf6-452d-84ee-fe6d7cb377df None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:23.142277 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ab6e5abb-faf6-452d-84ee-fe6d7cb377df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:23.150899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1408/2786] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:19:23.152927 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354a468-4c4e-42f1-b684-54be12611e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8ac27527654484b8e201d7b47fffd9, user_id=885b6e6f9613498baf0e071d32acd095, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.155160 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e5abb-faf6-452d-84ee-fe6d7cb377df None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.155814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354a468-4c4e-42f1-b684-54be12611e86 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.165997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e5abb-faf6-452d-84ee-fe6d7cb377df None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.169790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1379/2787] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:23 2022] PUT /identity/v3/projects/2a8ac27527654484b8e201d7b47fffd9/users/885b6e6f9613498baf0e071d32acd095/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:19:23.174136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354a468-4c4e-42f1-b684-54be12611e86 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.225735 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa300f5-2726-401b-9f2b-e42169135366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.226409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa300f5-2726-401b-9f2b-e42169135366 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.226693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa300f5-2726-401b-9f2b-e42169135366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.226984 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa300f5-2726-401b-9f2b-e42169135366 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.227635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa300f5-2726-401b-9f2b-e42169135366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:19:23.227932 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa300f5-2726-401b-9f2b-e42169135366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.229516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa300f5-2726-401b-9f2b-e42169135366 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.237757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354a468-4c4e-42f1-b684-54be12611e86 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.242663 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1409/2788] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 17:19:23.243004 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7931ee-718f-4d2a-b903-9c2a32d4472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.243616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7931ee-718f-4d2a-b903-9c2a32d4472d admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.243924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7931ee-718f-4d2a-b903-9c2a32d4472d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.244197 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7931ee-718f-4d2a-b903-9c2a32d4472d admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.244842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7931ee-718f-4d2a-b903-9c2a32d4472d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.246739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7931ee-718f-4d2a-b903-9c2a32d4472d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.247103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa300f5-2726-401b-9f2b-e42169135366 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.257647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1380/2789] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:23.257987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa300f5-2726-401b-9f2b-e42169135366 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.261698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7931ee-718f-4d2a-b903-9c2a32d4472d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.262537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3718894-7ebf-45be-9331-528b91380cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.263163 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3718894-7ebf-45be-9331-528b91380cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.263440 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3718894-7ebf-45be-9331-528b91380cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.263708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3718894-7ebf-45be-9331-528b91380cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.264343 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3718894-7ebf-45be-9331-528b91380cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.266815 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3718894-7ebf-45be-9331-528b91380cb1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.272169 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7931ee-718f-4d2a-b903-9c2a32d4472d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.276861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889a511-cc20-4cf3-856e-4459471c6680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.277532 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889a511-cc20-4cf3-856e-4459471c6680 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.277796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889a511-cc20-4cf3-856e-4459471c6680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.278070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889a511-cc20-4cf3-856e-4459471c6680 admin admin] PATH_INFO: `/v3/projects/2a8ac27527654484b8e201d7b47fffd9/users/885b6e6f9613498baf0e071d32acd095/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.280056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889a511-cc20-4cf3-856e-4459471c6680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8ac27527654484b8e201d7b47fffd9, user_id=885b6e6f9613498baf0e071d32acd095, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.283285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889a511-cc20-4cf3-856e-4459471c6680 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.298134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1410/2791] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:19:23 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:19:23.299534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3718894-7ebf-45be-9331-528b91380cb1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.303903 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1381/2791] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:23 2022] PUT /identity/v3/projects/2a8ac27527654484b8e201d7b47fffd9/users/885b6e6f9613498baf0e071d32acd095/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:23.305302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fe67d-e27e-4073-b69f-bd2a5c7e200b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.305592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fe67d-e27e-4073-b69f-bd2a5c7e200b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.305900 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fe67d-e27e-4073-b69f-bd2a5c7e200b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.335902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3b1fe67d-e27e-4073-b69f-bd2a5c7e200b None None] MFA Rules not processed for user `885b6e6f9613498baf0e071d32acd095`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:23.337255 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3b1fe67d-e27e-4073-b69f-bd2a5c7e200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:23.346893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:23.369470 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:23.376297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3718894-7ebf-45be-9331-528b91380cb1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.385476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745f662-b27b-44bd-96e3-19f5f30601e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.386290 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745f662-b27b-44bd-96e3-19f5f30601e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.386723 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745f662-b27b-44bd-96e3-19f5f30601e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.387212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745f662-b27b-44bd-96e3-19f5f30601e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.388273 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745f662-b27b-44bd-96e3-19f5f30601e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.391987 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745f662-b27b-44bd-96e3-19f5f30601e7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.414801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1411/2792] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:19:23 2022] POST /identity/v3/users => generated 285 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:19:23.418790 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745f662-b27b-44bd-96e3-19f5f30601e7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.428330 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1382/2793] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:23.429261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745f662-b27b-44bd-96e3-19f5f30601e7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.434843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412587fd-c1d2-4808-8be7-826f4137e48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.435029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412587fd-c1d2-4808-8be7-826f4137e48b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.435252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412587fd-c1d2-4808-8be7-826f4137e48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.435523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412587fd-c1d2-4808-8be7-826f4137e48b admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.436164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412587fd-c1d2-4808-8be7-826f4137e48b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.437749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412587fd-c1d2-4808-8be7-826f4137e48b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.440226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fe67d-e27e-4073-b69f-bd2a5c7e200b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.441710 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3b1fe67d-e27e-4073-b69f-bd2a5c7e200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:23.448292 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1412/2794] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:23.452226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412587fd-c1d2-4808-8be7-826f4137e48b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.462391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412587fd-c1d2-4808-8be7-826f4137e48b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.467333 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc1530-659f-4c4d-bc2e-49dbac72c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.468102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1530-659f-4c4d-bc2e-49dbac72c07c admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.468389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1530-659f-4c4d-bc2e-49dbac72c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.468750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1530-659f-4c4d-bc2e-49dbac72c07c admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.469893 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc1530-659f-4c4d-bc2e-49dbac72c07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.471507 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc1530-659f-4c4d-bc2e-49dbac72c07c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.486544 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1413/2795] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:19:23.487612 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc1530-659f-4c4d-bc2e-49dbac72c07c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.497496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc1530-659f-4c4d-bc2e-49dbac72c07c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.502778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d4f9e-62c3-449c-a459-e76fcd2fe056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.503445 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d4f9e-62c3-449c-a459-e76fcd2fe056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.503730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d4f9e-62c3-449c-a459-e76fcd2fe056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.503995 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d4f9e-62c3-449c-a459-e76fcd2fe056 admin admin] PATH_INFO: `/v3/projects/a1897d1d20e74e7e80577a208500a701/users/34185e019f99456abb7b5ce47282f07f/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.504445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fe67d-e27e-4073-b69f-bd2a5c7e200b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.508774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975135d-bebb-4c40-a7d6-e67dfe7c3536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.509456 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975135d-bebb-4c40-a7d6-e67dfe7c3536 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.509808 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975135d-bebb-4c40-a7d6-e67dfe7c3536 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.510120 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975135d-bebb-4c40-a7d6-e67dfe7c3536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.519264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d4f9e-62c3-449c-a459-e76fcd2fe056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1897d1d20e74e7e80577a208500a701, user_id=34185e019f99456abb7b5ce47282f07f, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.520414 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975135d-bebb-4c40-a7d6-e67dfe7c3536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.521839 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975135d-bebb-4c40-a7d6-e67dfe7c3536 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.522003 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d4f9e-62c3-449c-a459-e76fcd2fe056 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.532863 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1383/2796] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:19:23.544478 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1414/2797] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:19:23 2022] PUT /identity/v3/projects/a1897d1d20e74e7e80577a208500a701/users/34185e019f99456abb7b5ce47282f07f/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:19:23.557453 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36c5d-a8a7-4a84-9b4b-ccb12e7045e3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.557741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36c5d-a8a7-4a84-9b4b-ccb12e7045e3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.558080 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36c5d-a8a7-4a84-9b4b-ccb12e7045e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.602613 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-35f36c5d-a8a7-4a84-9b4b-ccb12e7045e3 None None] MFA Rules not processed for user `34185e019f99456abb7b5ce47282f07f`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:19:23.603882 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-35f36c5d-a8a7-4a84-9b4b-ccb12e7045e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:23.622900 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d4f9e-62c3-449c-a459-e76fcd2fe056 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.626115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5c5d4f9e-62c3-449c-a459-e76fcd2fe056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:23.685433 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1384/2798] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:19:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:19:23.691955 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d4f9e-62c3-449c-a459-e76fcd2fe056 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.695708 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f0ac6-24cf-439b-8178-3d4134fdc49c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.696342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f0ac6-24cf-439b-8178-3d4134fdc49c service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.696628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f0ac6-24cf-439b-8178-3d4134fdc49c service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.696901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f0ac6-24cf-439b-8178-3d4134fdc49c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.698620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f36c5d-a8a7-4a84-9b4b-ccb12e7045e3 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:19:23.698766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3a9f0ac6-24cf-439b-8178-3d4134fdc49c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:23.700129 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-35f36c5d-a8a7-4a84-9b4b-ccb12e7045e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:19:23.741186 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:23.760720 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f36c5d-a8a7-4a84-9b4b-ccb12e7045e3 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:19:23.760970 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f0ac6-24cf-439b-8178-3d4134fdc49c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.762216 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f0ac6-24cf-439b-8178-3d4134fdc49c service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.764921 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe28460-1a41-4622-b92b-94c13e5759b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6103e9a612c84a9ea7d21cc5128fa40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:19:23.766341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe28460-1a41-4622-b92b-94c13e5759b3 service glance] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:19:23.766855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe28460-1a41-4622-b92b-94c13e5759b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:19:23.767179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe28460-1a41-4622-b92b-94c13e5759b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:19:23.772493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1415/2799] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:19:23.776592 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe28460-1a41-4622-b92b-94c13e5759b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:19:23.778010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe28460-1a41-4622-b92b-94c13e5759b3 service glance] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:19:23.787214 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1385/2800] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:19:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:19:33.768479 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=71,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:19:33.783135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=9 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:16.874927 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f0ac6-24cf-439b-8178-3d4134fdc49c service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:16.943734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f0ac6-24cf-439b-8178-3d4134fdc49c service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:16.949007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:16.949669 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:16.950032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:16.950340 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] PATH_INFO: `/v3/users/34185e019f99456abb7b5ce47282f07f` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:16.951579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34185e019f99456abb7b5ce47282f07f)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:16.953616 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:17.009451 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34185e019f99456abb7b5ce47282f07f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:17.022393 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34185e019f99456abb7b5ce47282f07f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:17.030383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34185e019f99456abb7b5ce47282f07f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:17.040493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34185e019f99456abb7b5ce47282f07f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:17.041199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34185e019f99456abb7b5ce47282f07f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:17.048266 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1416/2801] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:20:16 2022] DELETE /identity/v3/users/34185e019f99456abb7b5ce47282f07f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:17.053016 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402127dc-1284-4eb0-9749-acf736799cc4 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:17.053497 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402127dc-1284-4eb0-9749-acf736799cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:17.053607 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402127dc-1284-4eb0-9749-acf736799cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:17.072785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-402127dc-1284-4eb0-9749-acf736799cc4 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:17.073980 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-402127dc-1284-4eb0-9749-acf736799cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:17.136658 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1386/2802] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:17.147327 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:17.149526 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:17.212834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073058e0-49d4-4212-a8cd-e6e9bdda63f8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:17.217111 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96fb45-ae62-49fc-a734-a1cffa1a8560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:17.217718 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96fb45-ae62-49fc-a734-a1cffa1a8560 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:17.217996 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96fb45-ae62-49fc-a734-a1cffa1a8560 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:17.218265 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96fb45-ae62-49fc-a734-a1cffa1a8560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:17.229454 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96fb45-ae62-49fc-a734-a1cffa1a8560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:17.230872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96fb45-ae62-49fc-a734-a1cffa1a8560 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:17.241229 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1417/2803] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:20:17.563972 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402127dc-1284-4eb0-9749-acf736799cc4 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:17.565470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-402127dc-1284-4eb0-9749-acf736799cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:17.579149 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402127dc-1284-4eb0-9749-acf736799cc4 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:17.584035 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:17.584537 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:17.584819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:17.585095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] PATH_INFO: `/v3/projects/a1897d1d20e74e7e80577a208500a701` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:17.586981 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1897d1d20e74e7e80577a208500a701)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:17.588778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:17.640667 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1897d1d20e74e7e80577a208500a701', 'request_id': 'req-3fda5e57-0312-4eeb-a769-b310d16fa576'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:17.646097 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1897d1d20e74e7e80577a208500a701', 'request_id': 'req-3fda5e57-0312-4eeb-a769-b310d16fa576'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:17.652373 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1897d1d20e74e7e80577a208500a701', 'request_id': 'req-3fda5e57-0312-4eeb-a769-b310d16fa576'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:17.665051 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] The token cache is being invalidate because project a1897d1d20e74e7e80577a208500a701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:20:17.665341 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:17.665953 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3fda5e57-0312-4eeb-a769-b310d16fa576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:17.667464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1387/2804] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:20:17 2022] DELETE /identity/v3/projects/a1897d1d20e74e7e80577a208500a701 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:27.236549 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:27.661977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:31.413516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96fb45-ae62-49fc-a734-a1cffa1a8560 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:31.415082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5f96fb45-ae62-49fc-a734-a1cffa1a8560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:31.429456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96fb45-ae62-49fc-a734-a1cffa1a8560 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:31.495093 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:31.496077 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:31.496565 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:31.497037 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] PATH_INFO: `/v3/users/e710fb55566948d3b7091322fd25f3da` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:31.498882 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e710fb55566948d3b7091322fd25f3da)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:31.500829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:31.552231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e710fb55566948d3b7091322fd25f3da'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:31.568154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e710fb55566948d3b7091322fd25f3da'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:31.579538 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e710fb55566948d3b7091322fd25f3da'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:31.587700 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e710fb55566948d3b7091322fd25f3da'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:31.588320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e710fb55566948d3b7091322fd25f3da'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:31.595409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1418/2805] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:20:31 2022] DELETE /identity/v3/users/e710fb55566948d3b7091322fd25f3da => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:20:31.599853 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4704-70a0-4156-bf9a-bc7a9aa663ae None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:31.600322 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4704-70a0-4156-bf9a-bc7a9aa663ae None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:31.600451 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4704-70a0-4156-bf9a-bc7a9aa663ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:31.619820 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-588a4704-70a0-4156-bf9a-bc7a9aa663ae None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:31.621029 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-588a4704-70a0-4156-bf9a-bc7a9aa663ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:31.679625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1388/2806] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:31.689262 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:31.690678 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:31.748766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb421bf-2bc5-455a-b04c-55521f813045 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:31.752787 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9950a-e1f5-4d9c-9c8b-3a00add33376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:31.753407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9950a-e1f5-4d9c-9c8b-3a00add33376 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:31.753703 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9950a-e1f5-4d9c-9c8b-3a00add33376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:31.753991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9950a-e1f5-4d9c-9c8b-3a00add33376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:31.764743 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9950a-e1f5-4d9c-9c8b-3a00add33376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:31.766555 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9950a-e1f5-4d9c-9c8b-3a00add33376 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:31.777029 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1419/2807] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:20:32.046776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a4704-70a0-4156-bf9a-bc7a9aa663ae None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:32.048239 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-588a4704-70a0-4156-bf9a-bc7a9aa663ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:32.063230 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a4704-70a0-4156-bf9a-bc7a9aa663ae None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:32.069024 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:32.069669 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:32.069985 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:32.070207 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] PATH_INFO: `/v3/projects/3d75fcb58d0c45f7b7b1dd70e979858f` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:32.071490 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d75fcb58d0c45f7b7b1dd70e979858f)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:32.073381 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:32.121130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d75fcb58d0c45f7b7b1dd70e979858f', 'request_id': 'req-a8029690-5d6a-4f6f-bba8-70eb4b15897d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.126750 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d75fcb58d0c45f7b7b1dd70e979858f', 'request_id': 'req-a8029690-5d6a-4f6f-bba8-70eb4b15897d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.132955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d75fcb58d0c45f7b7b1dd70e979858f', 'request_id': 'req-a8029690-5d6a-4f6f-bba8-70eb4b15897d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.144917 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] The token cache is being invalidate because project 3d75fcb58d0c45f7b7b1dd70e979858f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:20:32.145204 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.145829 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.147369 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1389/2808] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:20:32 2022] DELETE /identity/v3/projects/3d75fcb58d0c45f7b7b1dd70e979858f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:32.152088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9950a-e1f5-4d9c-9c8b-3a00add33376 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:32.166096 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9950a-e1f5-4d9c-9c8b-3a00add33376 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:32.188009 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:32.229227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb835af7-d33c-4661-b904-6b65080ea523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:32.229853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb835af7-d33c-4661-b904-6b65080ea523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:32.230126 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb835af7-d33c-4661-b904-6b65080ea523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:32.230395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb835af7-d33c-4661-b904-6b65080ea523 admin admin] PATH_INFO: `/v3/users/885b6e6f9613498baf0e071d32acd095` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:32.231534 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb835af7-d33c-4661-b904-6b65080ea523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=885b6e6f9613498baf0e071d32acd095)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:32.233427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb835af7-d33c-4661-b904-6b65080ea523 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:32.281432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb835af7-d33c-4661-b904-6b65080ea523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '885b6e6f9613498baf0e071d32acd095'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.293552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb835af7-d33c-4661-b904-6b65080ea523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '885b6e6f9613498baf0e071d32acd095'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.300355 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb835af7-d33c-4661-b904-6b65080ea523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '885b6e6f9613498baf0e071d32acd095'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.307924 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb835af7-d33c-4661-b904-6b65080ea523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '885b6e6f9613498baf0e071d32acd095'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.308575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb835af7-d33c-4661-b904-6b65080ea523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '885b6e6f9613498baf0e071d32acd095'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.314426 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1420/2809] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:20:32 2022] DELETE /identity/v3/users/885b6e6f9613498baf0e071d32acd095 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:20:32.587654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:32.589221 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:32.605134 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8029690-5d6a-4f6f-bba8-70eb4b15897d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:32.677114 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:32.677674 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:32.678246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:32.678378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] PATH_INFO: `/v3/projects/2a8ac27527654484b8e201d7b47fffd9` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:32.679551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8ac27527654484b8e201d7b47fffd9)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:32.683918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:32.720654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:32.740564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8ac27527654484b8e201d7b47fffd9', 'request_id': 'req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.747628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8ac27527654484b8e201d7b47fffd9', 'request_id': 'req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.756979 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8ac27527654484b8e201d7b47fffd9', 'request_id': 'req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.771764 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] The token cache is being invalidate because project 2a8ac27527654484b8e201d7b47fffd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:20:32.772061 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.772695 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e2fb110b-0dd4-4bd0-b736-df64cfb36f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:32.774191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1390/2810] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:20:32 2022] DELETE /identity/v3/projects/2a8ac27527654484b8e201d7b47fffd9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:32.779187 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ade4f7-7340-4c57-a79b-5b79615ba46d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:32.779744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ade4f7-7340-4c57-a79b-5b79615ba46d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:32.779744 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ade4f7-7340-4c57-a79b-5b79615ba46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:32.799579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-95ade4f7-7340-4c57-a79b-5b79615ba46d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:32.801042 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-95ade4f7-7340-4c57-a79b-5b79615ba46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:32.862285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1421/2811] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:32.870795 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cda8f5-5fae-4b32-b4c6-20e96bc67eff None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:32.871236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cda8f5-5fae-4b32-b4c6-20e96bc67eff None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:32.871436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cda8f5-5fae-4b32-b4c6-20e96bc67eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:32.892148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-94cda8f5-5fae-4b32-b4c6-20e96bc67eff None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:32.893513 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-94cda8f5-5fae-4b32-b4c6-20e96bc67eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:32.901455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1391/2812] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:32.905819 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ade4f7-7340-4c57-a79b-5b79615ba46d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:32.915794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ade4f7-7340-4c57-a79b-5b79615ba46d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:32.921491 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703b1ef-f407-44e6-bd8f-e8685610664b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:32.922186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703b1ef-f407-44e6-bd8f-e8685610664b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:32.922471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703b1ef-f407-44e6-bd8f-e8685610664b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:32.922767 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703b1ef-f407-44e6-bd8f-e8685610664b admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:32.923473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703b1ef-f407-44e6-bd8f-e8685610664b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:20:32.923781 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703b1ef-f407-44e6-bd8f-e8685610664b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:32.925375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703b1ef-f407-44e6-bd8f-e8685610664b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:32.938627 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1422/2813] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:20:32.943642 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cda8f5-5fae-4b32-b4c6-20e96bc67eff None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:32.953346 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cda8f5-5fae-4b32-b4c6-20e96bc67eff None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:32.958450 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2592117-90b0-4192-ba32-42adac3f73b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:32.959111 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2592117-90b0-4192-ba32-42adac3f73b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:32.959394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2592117-90b0-4192-ba32-42adac3f73b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:32.959668 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2592117-90b0-4192-ba32-42adac3f73b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:32.960306 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2592117-90b0-4192-ba32-42adac3f73b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:32.962315 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2592117-90b0-4192-ba32-42adac3f73b1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:32.992217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1392/2814] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:20:32 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:20:32.997018 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703b1ef-f407-44e6-bd8f-e8685610664b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:33.007353 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703b1ef-f407-44e6-bd8f-e8685610664b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:33.035242 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:33.069221 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a038dbf-288d-4631-a4fa-7902c3471a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:33.069880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a038dbf-288d-4631-a4fa-7902c3471a3f admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.070165 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a038dbf-288d-4631-a4fa-7902c3471a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.070443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a038dbf-288d-4631-a4fa-7902c3471a3f admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.071122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a038dbf-288d-4631-a4fa-7902c3471a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:33.073034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a038dbf-288d-4631-a4fa-7902c3471a3f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:33.093866 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1423/2815] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:20:32 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:20:33.098002 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2592117-90b0-4192-ba32-42adac3f73b1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:33.108869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2592117-90b0-4192-ba32-42adac3f73b1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:33.113649 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fca931-0334-4682-8469-464ad7452dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:33.114245 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fca931-0334-4682-8469-464ad7452dea admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.114525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fca931-0334-4682-8469-464ad7452dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.114799 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fca931-0334-4682-8469-464ad7452dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.115482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fca931-0334-4682-8469-464ad7452dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:33.117046 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fca931-0334-4682-8469-464ad7452dea admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:33.127932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1393/2816] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:20:33.132080 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a038dbf-288d-4631-a4fa-7902c3471a3f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:33.142183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a038dbf-288d-4631-a4fa-7902c3471a3f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:33.146962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303f949-1e3e-44a1-b3ab-55359e30623d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:33.147579 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303f949-1e3e-44a1-b3ab-55359e30623d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.147862 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303f949-1e3e-44a1-b3ab-55359e30623d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.148151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303f949-1e3e-44a1-b3ab-55359e30623d admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.148772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303f949-1e3e-44a1-b3ab-55359e30623d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:33.150345 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303f949-1e3e-44a1-b3ab-55359e30623d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:33.161097 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1424/2817] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:20:33.165455 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fca931-0334-4682-8469-464ad7452dea admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:33.177410 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fca931-0334-4682-8469-464ad7452dea admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:33.182141 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af08c5-bb52-49d9-aefd-a3c896a36afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:33.183807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af08c5-bb52-49d9-aefd-a3c896a36afb admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.184086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af08c5-bb52-49d9-aefd-a3c896a36afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.184359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af08c5-bb52-49d9-aefd-a3c896a36afb admin admin] PATH_INFO: `/v3/projects/8a3643aa82fe484795aa3b6b1a9e06dc/users/2154d6559fc340ca95507a2a81b10356/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.200523 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af08c5-bb52-49d9-aefd-a3c896a36afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a3643aa82fe484795aa3b6b1a9e06dc, user_id=2154d6559fc340ca95507a2a81b10356, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:33.203130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af08c5-bb52-49d9-aefd-a3c896a36afb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:33.217541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1394/2818] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:20:33 2022] PUT /identity/v3/projects/8a3643aa82fe484795aa3b6b1a9e06dc/users/2154d6559fc340ca95507a2a81b10356/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:33.228016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749a66e-c534-4143-9a84-91c9b93477b7 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.228295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749a66e-c534-4143-9a84-91c9b93477b7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.228593 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749a66e-c534-4143-9a84-91c9b93477b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.257859 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5749a66e-c534-4143-9a84-91c9b93477b7 None None] MFA Rules not processed for user `2154d6559fc340ca95507a2a81b10356`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:33.259114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5749a66e-c534-4143-9a84-91c9b93477b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:33.326364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1425/2819] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:33.330913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af08c5-bb52-49d9-aefd-a3c896a36afb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:33.392945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af08c5-bb52-49d9-aefd-a3c896a36afb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:33.397683 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedbdf6-f6b5-44de-8e6e-09cf318d9c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:33.398296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedbdf6-f6b5-44de-8e6e-09cf318d9c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.398579 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedbdf6-f6b5-44de-8e6e-09cf318d9c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.398852 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedbdf6-f6b5-44de-8e6e-09cf318d9c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.399526 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedbdf6-f6b5-44de-8e6e-09cf318d9c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:33.401978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedbdf6-f6b5-44de-8e6e-09cf318d9c32 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:33.431011 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1395/2820] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:20:33 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:20:33.435159 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749a66e-c534-4143-9a84-91c9b93477b7 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:33.445857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749a66e-c534-4143-9a84-91c9b93477b7 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:33.505462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07594dfb-e63b-4c67-afdf-462790098f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:33.505982 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07594dfb-e63b-4c67-afdf-462790098f9b admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.506256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07594dfb-e63b-4c67-afdf-462790098f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.506524 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07594dfb-e63b-4c67-afdf-462790098f9b admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.507183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07594dfb-e63b-4c67-afdf-462790098f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:33.509078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07594dfb-e63b-4c67-afdf-462790098f9b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:33.530650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1426/2821] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:20:33 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:20:33.534807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedbdf6-f6b5-44de-8e6e-09cf318d9c32 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:33.546738 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedbdf6-f6b5-44de-8e6e-09cf318d9c32 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:33.552670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae6293-90b3-493c-a100-bf8aec5c3bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:33.553470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae6293-90b3-493c-a100-bf8aec5c3bed admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.553855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae6293-90b3-493c-a100-bf8aec5c3bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.554233 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae6293-90b3-493c-a100-bf8aec5c3bed admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.555012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae6293-90b3-493c-a100-bf8aec5c3bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:33.556878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae6293-90b3-493c-a100-bf8aec5c3bed admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:33.568434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1396/2822] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:20:33.572659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07594dfb-e63b-4c67-afdf-462790098f9b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:33.582548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07594dfb-e63b-4c67-afdf-462790098f9b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:33.587325 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0905d9-ac8b-4749-9a8e-69d9f47fe5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:33.587989 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0905d9-ac8b-4749-9a8e-69d9f47fe5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.588274 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0905d9-ac8b-4749-9a8e-69d9f47fe5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.589196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0905d9-ac8b-4749-9a8e-69d9f47fe5fb admin admin] PATH_INFO: `/v3/projects/7e97e7e06ac04b7e89a5d5ee01e2e5c0/users/521c2d92656c4276b5e424a8e0aaa8ed/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.605801 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0905d9-ac8b-4749-9a8e-69d9f47fe5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e97e7e06ac04b7e89a5d5ee01e2e5c0, user_id=521c2d92656c4276b5e424a8e0aaa8ed, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:33.608469 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0905d9-ac8b-4749-9a8e-69d9f47fe5fb admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:33.621495 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1427/2823] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:20:33 2022] PUT /identity/v3/projects/7e97e7e06ac04b7e89a5d5ee01e2e5c0/users/521c2d92656c4276b5e424a8e0aaa8ed/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:20:33.625505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae6293-90b3-493c-a100-bf8aec5c3bed admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:33.636627 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae6293-90b3-493c-a100-bf8aec5c3bed admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:33.659857 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:33.705364 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba51653-e413-4915-acdb-af982b14df02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:33.705963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51653-e413-4915-acdb-af982b14df02 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.706244 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51653-e413-4915-acdb-af982b14df02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.706514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51653-e413-4915-acdb-af982b14df02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.707178 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba51653-e413-4915-acdb-af982b14df02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:33.708753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba51653-e413-4915-acdb-af982b14df02 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:33.718977 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1397/2824] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:20:33.723186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0905d9-ac8b-4749-9a8e-69d9f47fe5fb admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:33.728721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:33.735120 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0905d9-ac8b-4749-9a8e-69d9f47fe5fb admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:33.740138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26ae80-e4f6-44da-97bb-19e6c5455cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:33.740782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26ae80-e4f6-44da-97bb-19e6c5455cac admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.741062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26ae80-e4f6-44da-97bb-19e6c5455cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.741332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26ae80-e4f6-44da-97bb-19e6c5455cac admin admin] PATH_INFO: `/v3/projects/7e97e7e06ac04b7e89a5d5ee01e2e5c0/users/521c2d92656c4276b5e424a8e0aaa8ed/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.743366 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26ae80-e4f6-44da-97bb-19e6c5455cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e97e7e06ac04b7e89a5d5ee01e2e5c0, user_id=521c2d92656c4276b5e424a8e0aaa8ed, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:33.746150 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26ae80-e4f6-44da-97bb-19e6c5455cac admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:33.768552 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1428/2825] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:20:33 2022] PUT /identity/v3/projects/7e97e7e06ac04b7e89a5d5ee01e2e5c0/users/521c2d92656c4276b5e424a8e0aaa8ed/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:33.770700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2e3c7-b6b8-4a73-9a66-ecd2d692887c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.771039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2e3c7-b6b8-4a73-9a66-ecd2d692887c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.771314 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2e3c7-b6b8-4a73-9a66-ecd2d692887c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.800551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bee2e3c7-b6b8-4a73-9a66-ecd2d692887c None None] MFA Rules not processed for user `521c2d92656c4276b5e424a8e0aaa8ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:33.802047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bee2e3c7-b6b8-4a73-9a66-ecd2d692887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:33.882139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1398/2826] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:33.894389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26ae80-e4f6-44da-97bb-19e6c5455cac admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:33.896138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ef26ae80-e4f6-44da-97bb-19e6c5455cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:33.959930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26ae80-e4f6-44da-97bb-19e6c5455cac admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:33.964348 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3516d-7eaa-440a-b173-9dc429656cf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:33.964945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3516d-7eaa-440a-b173-9dc429656cf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:33.965222 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3516d-7eaa-440a-b173-9dc429656cf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:33.965490 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3516d-7eaa-440a-b173-9dc429656cf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:33.974737 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3516d-7eaa-440a-b173-9dc429656cf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:33.976212 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3516d-7eaa-440a-b173-9dc429656cf4 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:33.986225 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1429/2827] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:20:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:20:34.415916 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2e3c7-b6b8-4a73-9a66-ecd2d692887c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:34.476850 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2e3c7-b6b8-4a73-9a66-ecd2d692887c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:34.482165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:34.482798 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:34.483115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:34.483391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] PATH_INFO: `/v3/users/2154d6559fc340ca95507a2a81b10356` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:34.484514 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2154d6559fc340ca95507a2a81b10356)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:34.486464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:34.532761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2154d6559fc340ca95507a2a81b10356'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:34.533816 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:34.547679 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2154d6559fc340ca95507a2a81b10356'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:34.554864 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2154d6559fc340ca95507a2a81b10356'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:34.555584 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2154d6559fc340ca95507a2a81b10356'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:34.578496 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2154d6559fc340ca95507a2a81b10356'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:34.586569 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1399/2828] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:20:34 2022] DELETE /identity/v3/users/2154d6559fc340ca95507a2a81b10356 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:34.594148 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7572e-1d82-4d68-a75d-e10719326962 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:34.594433 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7572e-1d82-4d68-a75d-e10719326962 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:34.594759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7572e-1d82-4d68-a75d-e10719326962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:34.614082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-58a7572e-1d82-4d68-a75d-e10719326962 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:34.615319 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-58a7572e-1d82-4d68-a75d-e10719326962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:34.672749 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1430/2829] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:34.683236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:34.684703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:34.740860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382c1b5-3ace-4d6c-8d52-009bae579c1a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:34.744505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75424ef8-79d2-4543-b250-f111b53409f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:34.745647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75424ef8-79d2-4543-b250-f111b53409f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:34.745924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75424ef8-79d2-4543-b250-f111b53409f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:34.746202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75424ef8-79d2-4543-b250-f111b53409f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:34.755581 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75424ef8-79d2-4543-b250-f111b53409f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:34.757008 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75424ef8-79d2-4543-b250-f111b53409f9 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:34.767376 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1400/2830] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:20:34.915020 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7572e-1d82-4d68-a75d-e10719326962 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:34.917188 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-58a7572e-1d82-4d68-a75d-e10719326962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:34.929991 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7572e-1d82-4d68-a75d-e10719326962 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:34.935812 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:34.936447 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:34.936788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:34.937521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] PATH_INFO: `/v3/users/377ece01a4284b7aa3db6d58c1733c4c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:34.938764 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] RBAC: Authorizing `identity:delete_user(user_id=377ece01a4284b7aa3db6d58c1733c4c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:34.940657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:34.990211 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377ece01a4284b7aa3db6d58c1733c4c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.002394 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377ece01a4284b7aa3db6d58c1733c4c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.009913 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377ece01a4284b7aa3db6d58c1733c4c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.017543 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377ece01a4284b7aa3db6d58c1733c4c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.018385 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377ece01a4284b7aa3db6d58c1733c4c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.024583 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1431/2831] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:20:34 2022] DELETE /identity/v3/users/377ece01a4284b7aa3db6d58c1733c4c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:35.029698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae03f3c-d22b-4011-b1ec-a3dbc555910c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:35.029993 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae03f3c-d22b-4011-b1ec-a3dbc555910c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:35.030296 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae03f3c-d22b-4011-b1ec-a3dbc555910c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:35.051154 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5ae03f3c-d22b-4011-b1ec-a3dbc555910c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:35.052374 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5ae03f3c-d22b-4011-b1ec-a3dbc555910c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:35.056759 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:35.058190 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:35.065497 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:35.121437 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1401/2832] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:35.131735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae03f3c-d22b-4011-b1ec-a3dbc555910c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:35.133263 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5ae03f3c-d22b-4011-b1ec-a3dbc555910c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:35.138386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faf026-7812-40e6-8e4b-28ae63eb898b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:35.143054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:35.143690 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:35.143969 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:35.144240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] PATH_INFO: `/v3/projects/8a3643aa82fe484795aa3b6b1a9e06dc` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:35.145375 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a3643aa82fe484795aa3b6b1a9e06dc)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:35.147296 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:35.191761 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae03f3c-d22b-4011-b1ec-a3dbc555910c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:35.203041 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a3643aa82fe484795aa3b6b1a9e06dc', 'request_id': 'req-69497172-0841-4665-8278-d0be695d7b5b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.212666 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a3643aa82fe484795aa3b6b1a9e06dc', 'request_id': 'req-69497172-0841-4665-8278-d0be695d7b5b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.222253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a3643aa82fe484795aa3b6b1a9e06dc', 'request_id': 'req-69497172-0841-4665-8278-d0be695d7b5b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.223049 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:35.239407 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] The token cache is being invalidate because project 8a3643aa82fe484795aa3b6b1a9e06dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:20:35.239837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.240409 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.242499 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1432/2833] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:20:35 2022] DELETE /identity/v3/projects/8a3643aa82fe484795aa3b6b1a9e06dc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:20:35.246659 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:35.248031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:35.259055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dad38-6b93-413e-9bbc-f09e9c0d94ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:35.259661 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dad38-6b93-413e-9bbc-f09e9c0d94ba service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:35.259939 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dad38-6b93-413e-9bbc-f09e9c0d94ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:35.260686 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dad38-6b93-413e-9bbc-f09e9c0d94ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:35.262474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b37dad38-6b93-413e-9bbc-f09e9c0d94ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:35.314109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69497172-0841-4665-8278-d0be695d7b5b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:35.319631 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:35.320281 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:35.320550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:35.320947 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] PATH_INFO: `/v3/users/521c2d92656c4276b5e424a8e0aaa8ed` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:35.322125 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=521c2d92656c4276b5e424a8e0aaa8ed)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:35.324681 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:35.331023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37dad38-6b93-413e-9bbc-f09e9c0d94ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:35.332478 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37dad38-6b93-413e-9bbc-f09e9c0d94ba service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:35.343818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1402/2834] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:20:35.366820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:35.377317 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '521c2d92656c4276b5e424a8e0aaa8ed'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.391444 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '521c2d92656c4276b5e424a8e0aaa8ed'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.400030 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '521c2d92656c4276b5e424a8e0aaa8ed'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.408820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '521c2d92656c4276b5e424a8e0aaa8ed'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.409512 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '521c2d92656c4276b5e424a8e0aaa8ed'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.416840 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1433/2835] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:20:35 2022] DELETE /identity/v3/users/521c2d92656c4276b5e424a8e0aaa8ed => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:20:35.694774 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dad38-6b93-413e-9bbc-f09e9c0d94ba service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:35.696391 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b37dad38-6b93-413e-9bbc-f09e9c0d94ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:35.711869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dad38-6b93-413e-9bbc-f09e9c0d94ba service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:35.737633 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:35.739140 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:35.794336 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:35.794999 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:35.795459 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:35.795743 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] PATH_INFO: `/v3/projects/1d274d2bacf045dc97c1c92ce862a90d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:35.796889 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d274d2bacf045dc97c1c92ce862a90d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:35.798941 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:35.815063 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfff53f-d7b3-4cec-ae07-b6c5554d3161 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:35.819799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:35.820432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:35.820713 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:35.821007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] PATH_INFO: `/v3/projects/7e97e7e06ac04b7e89a5d5ee01e2e5c0` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:35.822115 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e97e7e06ac04b7e89a5d5ee01e2e5c0)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:35.824099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:35.848082 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d274d2bacf045dc97c1c92ce862a90d', 'request_id': 'req-69edd019-ad5d-450c-b0f3-32104b2398f4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.855246 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d274d2bacf045dc97c1c92ce862a90d', 'request_id': 'req-69edd019-ad5d-450c-b0f3-32104b2398f4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.862017 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d274d2bacf045dc97c1c92ce862a90d', 'request_id': 'req-69edd019-ad5d-450c-b0f3-32104b2398f4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.874227 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e97e7e06ac04b7e89a5d5ee01e2e5c0', 'request_id': 'req-449b9d7d-a257-4e31-a500-9acecd82316d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.874655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] The token cache is being invalidate because project 1d274d2bacf045dc97c1c92ce862a90d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:20:35.875006 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.875700 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-69edd019-ad5d-450c-b0f3-32104b2398f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.877355 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1403/2836] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:20:35 2022] DELETE /identity/v3/projects/1d274d2bacf045dc97c1c92ce862a90d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:35.882937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e97e7e06ac04b7e89a5d5ee01e2e5c0', 'request_id': 'req-449b9d7d-a257-4e31-a500-9acecd82316d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.889799 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e97e7e06ac04b7e89a5d5ee01e2e5c0', 'request_id': 'req-449b9d7d-a257-4e31-a500-9acecd82316d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.905696 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] The token cache is being invalidate because project 7e97e7e06ac04b7e89a5d5ee01e2e5c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:20:35.906002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.906657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-449b9d7d-a257-4e31-a500-9acecd82316d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:35.908238 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1434/2837] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:20:35 2022] DELETE /identity/v3/projects/7e97e7e06ac04b7e89a5d5ee01e2e5c0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:20:35.913732 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15815-15cc-4248-9ef4-3ca344d90b3f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:35.914202 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15815-15cc-4248-9ef4-3ca344d90b3f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:35.914574 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15815-15cc-4248-9ef4-3ca344d90b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:35.925601 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:35.938349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8af15815-15cc-4248-9ef4-3ca344d90b3f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:35.939978 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8af15815-15cc-4248-9ef4-3ca344d90b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:36.005874 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1404/2838] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:36.014505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556ae0-87d9-4486-a5a1-fb5bf36dc6fd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.015119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556ae0-87d9-4486-a5a1-fb5bf36dc6fd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.015239 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556ae0-87d9-4486-a5a1-fb5bf36dc6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.036997 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f7556ae0-87d9-4486-a5a1-fb5bf36dc6fd None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:36.038236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f7556ae0-87d9-4486-a5a1-fb5bf36dc6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:36.046510 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1435/2839] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:36.051447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af15815-15cc-4248-9ef4-3ca344d90b3f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.061976 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af15815-15cc-4248-9ef4-3ca344d90b3f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.066928 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce80f5-f637-4d05-923b-421547559a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.067583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce80f5-f637-4d05-923b-421547559a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.067856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce80f5-f637-4d05-923b-421547559a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.068123 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce80f5-f637-4d05-923b-421547559a9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.068778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce80f5-f637-4d05-923b-421547559a9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:20:36.069076 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce80f5-f637-4d05-923b-421547559a9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.070690 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce80f5-f637-4d05-923b-421547559a9a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.083962 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1405/2840] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:20:36.089286 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556ae0-87d9-4486-a5a1-fb5bf36dc6fd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.099151 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556ae0-87d9-4486-a5a1-fb5bf36dc6fd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.104101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482343c-fa50-43bd-adff-2b040b7e6013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.104817 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482343c-fa50-43bd-adff-2b040b7e6013 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.105183 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482343c-fa50-43bd-adff-2b040b7e6013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.105522 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482343c-fa50-43bd-adff-2b040b7e6013 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.106251 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482343c-fa50-43bd-adff-2b040b7e6013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.108268 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482343c-fa50-43bd-adff-2b040b7e6013 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.137318 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1436/2841] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:20:36 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:20:36.141460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce80f5-f637-4d05-923b-421547559a9a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.152108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce80f5-f637-4d05-923b-421547559a9a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.207604 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ed8c9-ad28-41d6-87da-6becb93873d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.208209 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ed8c9-ad28-41d6-87da-6becb93873d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.208487 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ed8c9-ad28-41d6-87da-6becb93873d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.208757 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ed8c9-ad28-41d6-87da-6becb93873d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.209395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ed8c9-ad28-41d6-87da-6becb93873d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.211342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ed8c9-ad28-41d6-87da-6becb93873d7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.233796 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1406/2842] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:20:36 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:20:36.237993 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482343c-fa50-43bd-adff-2b040b7e6013 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.248303 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482343c-fa50-43bd-adff-2b040b7e6013 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.252945 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbbf95-d9e9-4075-b3c4-6dc54674db32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.253541 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbbf95-d9e9-4075-b3c4-6dc54674db32 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.253821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbbf95-d9e9-4075-b3c4-6dc54674db32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.254094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbbf95-d9e9-4075-b3c4-6dc54674db32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.254741 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbbf95-d9e9-4075-b3c4-6dc54674db32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.256404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbbf95-d9e9-4075-b3c4-6dc54674db32 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.267001 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1437/2843] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:20:36.270975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ed8c9-ad28-41d6-87da-6becb93873d7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.281634 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ed8c9-ad28-41d6-87da-6becb93873d7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.286137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34dfbd-7efc-4f99-abd2-1083d667098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.286815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34dfbd-7efc-4f99-abd2-1083d667098a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.287056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34dfbd-7efc-4f99-abd2-1083d667098a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.287324 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34dfbd-7efc-4f99-abd2-1083d667098a admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.287952 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34dfbd-7efc-4f99-abd2-1083d667098a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.289544 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34dfbd-7efc-4f99-abd2-1083d667098a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.299965 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1407/2844] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:20:36.304384 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbbf95-d9e9-4075-b3c4-6dc54674db32 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.311193 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:36.316965 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbbf95-d9e9-4075-b3c4-6dc54674db32 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.324605 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c775c1d-47cf-456c-935b-693e9c59f09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.325542 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c775c1d-47cf-456c-935b-693e9c59f09b admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.326032 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c775c1d-47cf-456c-935b-693e9c59f09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.326437 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c775c1d-47cf-456c-935b-693e9c59f09b admin admin] PATH_INFO: `/v3/projects/55990502b1024f6ba8e196f328f91e98/users/d890817da8fb4d96bdb46680cf419c5d/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.341580 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c775c1d-47cf-456c-935b-693e9c59f09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55990502b1024f6ba8e196f328f91e98, user_id=d890817da8fb4d96bdb46680cf419c5d, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.344270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c775c1d-47cf-456c-935b-693e9c59f09b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.357240 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1438/2845] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:20:36 2022] PUT /identity/v3/projects/55990502b1024f6ba8e196f328f91e98/users/d890817da8fb4d96bdb46680cf419c5d/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:20:36.367540 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b38b4-3a54-4c44-a2a1-5eba88098774 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.367849 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b38b4-3a54-4c44-a2a1-5eba88098774 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.368152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b38b4-3a54-4c44-a2a1-5eba88098774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.396787 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9b0b38b4-3a54-4c44-a2a1-5eba88098774 None None] MFA Rules not processed for user `d890817da8fb4d96bdb46680cf419c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:36.397907 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9b0b38b4-3a54-4c44-a2a1-5eba88098774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:36.458058 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:36.462861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1408/2846] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:36.466820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c775c1d-47cf-456c-935b-693e9c59f09b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.527880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c775c1d-47cf-456c-935b-693e9c59f09b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.532506 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccce9d-b0ce-4d86-a025-a065ce4b70de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.533099 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccce9d-b0ce-4d86-a025-a065ce4b70de admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.533377 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccce9d-b0ce-4d86-a025-a065ce4b70de admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.533644 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccce9d-b0ce-4d86-a025-a065ce4b70de admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.534275 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccce9d-b0ce-4d86-a025-a065ce4b70de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.536167 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccce9d-b0ce-4d86-a025-a065ce4b70de admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.564648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1439/2847] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:20:36 2022] POST /identity/v3/projects => generated 391 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:20:36.568833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b38b4-3a54-4c44-a2a1-5eba88098774 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.580325 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b38b4-3a54-4c44-a2a1-5eba88098774 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.639783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913bf8c-9440-401f-9543-4b6c9ffea203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.640398 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913bf8c-9440-401f-9543-4b6c9ffea203 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.643219 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913bf8c-9440-401f-9543-4b6c9ffea203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.643512 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913bf8c-9440-401f-9543-4b6c9ffea203 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.644170 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913bf8c-9440-401f-9543-4b6c9ffea203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.646780 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913bf8c-9440-401f-9543-4b6c9ffea203 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.670269 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1409/2848] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:20:36 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:20:36.674329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccce9d-b0ce-4d86-a025-a065ce4b70de admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.685521 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccce9d-b0ce-4d86-a025-a065ce4b70de admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.690164 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a37574-90bf-43ca-bab0-60a0b089c84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.690758 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a37574-90bf-43ca-bab0-60a0b089c84b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.691161 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a37574-90bf-43ca-bab0-60a0b089c84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.691351 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a37574-90bf-43ca-bab0-60a0b089c84b admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.691999 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a37574-90bf-43ca-bab0-60a0b089c84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.693578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a37574-90bf-43ca-bab0-60a0b089c84b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.704224 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1440/2849] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:20:36.708389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913bf8c-9440-401f-9543-4b6c9ffea203 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.718784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913bf8c-9440-401f-9543-4b6c9ffea203 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.723676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ecc28-73e9-41fc-8301-daf87bf6e688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.724328 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ecc28-73e9-41fc-8301-daf87bf6e688 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.724611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ecc28-73e9-41fc-8301-daf87bf6e688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.724887 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ecc28-73e9-41fc-8301-daf87bf6e688 admin admin] PATH_INFO: `/v3/projects/e7ce5f7ad50c4acea74a61353e022df1/users/b05041729fba4e14ab51bd583fbdb582/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.740075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ecc28-73e9-41fc-8301-daf87bf6e688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ce5f7ad50c4acea74a61353e022df1, user_id=b05041729fba4e14ab51bd583fbdb582, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.742749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ecc28-73e9-41fc-8301-daf87bf6e688 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.757753 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1410/2850] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:20:36 2022] PUT /identity/v3/projects/e7ce5f7ad50c4acea74a61353e022df1/users/b05041729fba4e14ab51bd583fbdb582/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:20:36.762010 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a37574-90bf-43ca-bab0-60a0b089c84b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.772234 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a37574-90bf-43ca-bab0-60a0b089c84b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.828890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffc48c-1261-4a58-bc8d-78c4c9c19021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.829496 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffc48c-1261-4a58-bc8d-78c4c9c19021 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.829777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffc48c-1261-4a58-bc8d-78c4c9c19021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.830053 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffc48c-1261-4a58-bc8d-78c4c9c19021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.830685 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffc48c-1261-4a58-bc8d-78c4c9c19021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.832309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffc48c-1261-4a58-bc8d-78c4c9c19021 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.842156 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1441/2851] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:20:36.846339 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ecc28-73e9-41fc-8301-daf87bf6e688 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:36.857123 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ecc28-73e9-41fc-8301-daf87bf6e688 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:36.861771 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386ee24-de86-46ea-acbb-8a97cbf849db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:36.862419 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386ee24-de86-46ea-acbb-8a97cbf849db admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.862698 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386ee24-de86-46ea-acbb-8a97cbf849db admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.863045 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386ee24-de86-46ea-acbb-8a97cbf849db admin admin] PATH_INFO: `/v3/projects/e7ce5f7ad50c4acea74a61353e022df1/users/b05041729fba4e14ab51bd583fbdb582/roles/1aa220d46e7949729d20b04e4e0c838d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.865620 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386ee24-de86-46ea-acbb-8a97cbf849db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ce5f7ad50c4acea74a61353e022df1, user_id=b05041729fba4e14ab51bd583fbdb582, role_id=1aa220d46e7949729d20b04e4e0c838d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:36.868479 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386ee24-de86-46ea-acbb-8a97cbf849db admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:36.881108 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1411/2852] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:20:36 2022] PUT /identity/v3/projects/e7ce5f7ad50c4acea74a61353e022df1/users/b05041729fba4e14ab51bd583fbdb582/roles/1aa220d46e7949729d20b04e4e0c838d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:20:36.892261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d53738-c69a-4323-9e09-84c63930fb35 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:36.892551 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d53738-c69a-4323-9e09-84c63930fb35 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:36.892827 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d53738-c69a-4323-9e09-84c63930fb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:36.920936 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-07d53738-c69a-4323-9e09-84c63930fb35 None None] MFA Rules not processed for user `b05041729fba4e14ab51bd583fbdb582`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:36.922089 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-07d53738-c69a-4323-9e09-84c63930fb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:36.963836 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:36.995595 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1442/2853] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:36 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:37.006185 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386ee24-de86-46ea-acbb-8a97cbf849db admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:37.007895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e386ee24-de86-46ea-acbb-8a97cbf849db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:37.065784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386ee24-de86-46ea-acbb-8a97cbf849db admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:37.069481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895b1b5-496b-40a3-86f4-0507fa89a147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:37.070075 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895b1b5-496b-40a3-86f4-0507fa89a147 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:37.070356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895b1b5-496b-40a3-86f4-0507fa89a147 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:37.070630 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895b1b5-496b-40a3-86f4-0507fa89a147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:37.079712 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895b1b5-496b-40a3-86f4-0507fa89a147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:37.081130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895b1b5-496b-40a3-86f4-0507fa89a147 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:37.090973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1412/2854] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:20:37 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:20:37.137712 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d53738-c69a-4323-9e09-84c63930fb35 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:37.139134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-07d53738-c69a-4323-9e09-84c63930fb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:37.151482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d53738-c69a-4323-9e09-84c63930fb35 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:37.155154 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1424787-5d6e-4f99-824e-5177374b18e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:37.155742 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1424787-5d6e-4f99-824e-5177374b18e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:37.156016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1424787-5d6e-4f99-824e-5177374b18e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:37.156289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1424787-5d6e-4f99-824e-5177374b18e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:37.211834 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1424787-5d6e-4f99-824e-5177374b18e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:37.213778 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1424787-5d6e-4f99-824e-5177374b18e4 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:37.222550 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1443/2855] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:20:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:20:40.533950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895b1b5-496b-40a3-86f4-0507fa89a147 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:40.591975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:40.598875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895b1b5-496b-40a3-86f4-0507fa89a147 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:40.603420 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:40.604043 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:40.604323 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:40.604625 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] PATH_INFO: `/v3/users/d890817da8fb4d96bdb46680cf419c5d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:40.605777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d890817da8fb4d96bdb46680cf419c5d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:40.607739 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:40.663353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd890817da8fb4d96bdb46680cf419c5d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:40.675171 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd890817da8fb4d96bdb46680cf419c5d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:40.682329 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd890817da8fb4d96bdb46680cf419c5d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:40.683144 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd890817da8fb4d96bdb46680cf419c5d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:40.690436 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd890817da8fb4d96bdb46680cf419c5d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:40.696583 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1413/2856] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:20:40 2022] DELETE /identity/v3/users/d890817da8fb4d96bdb46680cf419c5d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:40.701721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb366c-8776-4a9f-9d90-058b0295cf27 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:40.702203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb366c-8776-4a9f-9d90-058b0295cf27 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:40.702336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb366c-8776-4a9f-9d90-058b0295cf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:40.722259 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-61fb366c-8776-4a9f-9d90-058b0295cf27 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:40.723498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-61fb366c-8776-4a9f-9d90-058b0295cf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:40.783393 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1444/2857] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:40.792926 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:40.794331 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:40.851834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f62af-9944-44d5-85cf-b9a1959de78f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:40.855628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5039bc0-d01b-41c2-a363-61c202062c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:40.856230 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5039bc0-d01b-41c2-a363-61c202062c9a service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:40.856505 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5039bc0-d01b-41c2-a363-61c202062c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:40.856776 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5039bc0-d01b-41c2-a363-61c202062c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:40.865713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5039bc0-d01b-41c2-a363-61c202062c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:40.867179 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5039bc0-d01b-41c2-a363-61c202062c9a service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:40.877313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1414/2858] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:20:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:20:41.149957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb366c-8776-4a9f-9d90-058b0295cf27 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:41.151493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-61fb366c-8776-4a9f-9d90-058b0295cf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:41.163641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb366c-8776-4a9f-9d90-058b0295cf27 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:41.169042 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:41.169676 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:41.169957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:41.170256 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] PATH_INFO: `/v3/projects/55990502b1024f6ba8e196f328f91e98` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:41.171427 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] RBAC: Authorizing `identity:delete_project(project_id=55990502b1024f6ba8e196f328f91e98)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:41.173297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:41.229957 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55990502b1024f6ba8e196f328f91e98', 'request_id': 'req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.236600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55990502b1024f6ba8e196f328f91e98', 'request_id': 'req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.242488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55990502b1024f6ba8e196f328f91e98', 'request_id': 'req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.251780 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:41.257803 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] The token cache is being invalidate because project 55990502b1024f6ba8e196f328f91e98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:20:41.258102 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.259376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.261002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1445/2859] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:20:41 2022] DELETE /identity/v3/projects/55990502b1024f6ba8e196f328f91e98 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:41.265036 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5039bc0-d01b-41c2-a363-61c202062c9a service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:41.275529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5039bc0-d01b-41c2-a363-61c202062c9a service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:41.333556 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8e0b1-b47a-4d48-a92f-7d937b56e88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:41.334105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8e0b1-b47a-4d48-a92f-7d937b56e88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:41.334385 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8e0b1-b47a-4d48-a92f-7d937b56e88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:41.334656 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8e0b1-b47a-4d48-a92f-7d937b56e88d admin admin] PATH_INFO: `/v3/users/b05041729fba4e14ab51bd583fbdb582` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:41.335826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8e0b1-b47a-4d48-a92f-7d937b56e88d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b05041729fba4e14ab51bd583fbdb582)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:41.337727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8e0b1-b47a-4d48-a92f-7d937b56e88d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:41.359068 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:41.384936 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-11c8e0b1-b47a-4d48-a92f-7d937b56e88d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b05041729fba4e14ab51bd583fbdb582'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.396600 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-11c8e0b1-b47a-4d48-a92f-7d937b56e88d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b05041729fba4e14ab51bd583fbdb582'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.403423 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-11c8e0b1-b47a-4d48-a92f-7d937b56e88d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b05041729fba4e14ab51bd583fbdb582'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.404121 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-11c8e0b1-b47a-4d48-a92f-7d937b56e88d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b05041729fba4e14ab51bd583fbdb582'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.411275 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-11c8e0b1-b47a-4d48-a92f-7d937b56e88d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b05041729fba4e14ab51bd583fbdb582'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.417734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1415/2860] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:20:41 2022] DELETE /identity/v3/users/b05041729fba4e14ab51bd583fbdb582 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:20:41.686682 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:41.688236 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:41.703418 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1a225-28d5-4bfa-936b-6aff5fc1d86b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:41.762423 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:41.763088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:41.763367 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:41.763639 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] PATH_INFO: `/v3/projects/e7ce5f7ad50c4acea74a61353e022df1` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:41.764763 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ce5f7ad50c4acea74a61353e022df1)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:41.766672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:41.813364 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ce5f7ad50c4acea74a61353e022df1', 'request_id': 'req-754a9559-62b4-4088-a1ba-1be46cfbda44'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.819408 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ce5f7ad50c4acea74a61353e022df1', 'request_id': 'req-754a9559-62b4-4088-a1ba-1be46cfbda44'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.825238 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ce5f7ad50c4acea74a61353e022df1', 'request_id': 'req-754a9559-62b4-4088-a1ba-1be46cfbda44'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.838203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] The token cache is being invalidate because project e7ce5f7ad50c4acea74a61353e022df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:20:41.838492 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.839634 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-754a9559-62b4-4088-a1ba-1be46cfbda44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:41.841122 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1446/2861] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:20:41 2022] DELETE /identity/v3/projects/e7ce5f7ad50c4acea74a61353e022df1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:41.845945 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a80d6c-a9c7-46d2-91d4-99e41298b620 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:41.846430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a80d6c-a9c7-46d2-91d4-99e41298b620 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:41.846552 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a80d6c-a9c7-46d2-91d4-99e41298b620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:41.866719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-98a80d6c-a9c7-46d2-91d4-99e41298b620 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:41.868027 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-98a80d6c-a9c7-46d2-91d4-99e41298b620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:41.933651 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1416/2862] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:41.942249 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c90d0-6333-4c00-93ea-a693f35b0a69 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:41.942561 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c90d0-6333-4c00-93ea-a693f35b0a69 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:41.942784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c90d0-6333-4c00-93ea-a693f35b0a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:41.963568 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c37c90d0-6333-4c00-93ea-a693f35b0a69 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:41.965525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c37c90d0-6333-4c00-93ea-a693f35b0a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:41.973747 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1447/2863] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:41.978085 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a80d6c-a9c7-46d2-91d4-99e41298b620 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:41.988095 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a80d6c-a9c7-46d2-91d4-99e41298b620 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:41.992719 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57342a-9c14-434c-95f0-ed06915dec3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:41.993311 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57342a-9c14-434c-95f0-ed06915dec3f admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:41.993617 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57342a-9c14-434c-95f0-ed06915dec3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:41.993888 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57342a-9c14-434c-95f0-ed06915dec3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:41.994560 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57342a-9c14-434c-95f0-ed06915dec3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:20:41.994855 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57342a-9c14-434c-95f0-ed06915dec3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:41.996467 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57342a-9c14-434c-95f0-ed06915dec3f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:42.008234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1417/2864] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:20:42.012823 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c90d0-6333-4c00-93ea-a693f35b0a69 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:42.022207 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c90d0-6333-4c00-93ea-a693f35b0a69 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:42.027914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15333095-edd4-47a9-8e21-70eff8bd983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:42.028504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15333095-edd4-47a9-8e21-70eff8bd983d admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:42.028784 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15333095-edd4-47a9-8e21-70eff8bd983d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:42.029057 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15333095-edd4-47a9-8e21-70eff8bd983d admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:42.029693 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15333095-edd4-47a9-8e21-70eff8bd983d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:42.031648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15333095-edd4-47a9-8e21-70eff8bd983d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:42.058632 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1448/2865] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:20:42 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:20:42.062792 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57342a-9c14-434c-95f0-ed06915dec3f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:42.072856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57342a-9c14-434c-95f0-ed06915dec3f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:42.129137 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f7de4-acfc-48b3-a2da-1bef3e36f9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:42.129660 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f7de4-acfc-48b3-a2da-1bef3e36f9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:42.129938 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f7de4-acfc-48b3-a2da-1bef3e36f9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:42.130211 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f7de4-acfc-48b3-a2da-1bef3e36f9e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:42.130834 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f7de4-acfc-48b3-a2da-1bef3e36f9e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:42.132763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f7de4-acfc-48b3-a2da-1bef3e36f9e3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:42.157087 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1418/2866] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:20:42 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:20:42.158347 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15333095-edd4-47a9-8e21-70eff8bd983d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:42.168783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15333095-edd4-47a9-8e21-70eff8bd983d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:42.173577 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a1f03-cef5-462a-b89f-adf3d9632da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:42.174170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a1f03-cef5-462a-b89f-adf3d9632da3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:42.174452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a1f03-cef5-462a-b89f-adf3d9632da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:42.174727 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a1f03-cef5-462a-b89f-adf3d9632da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:42.175397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a1f03-cef5-462a-b89f-adf3d9632da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:42.177243 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a1f03-cef5-462a-b89f-adf3d9632da3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:42.187308 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1449/2867] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:20:42.191249 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f7de4-acfc-48b3-a2da-1bef3e36f9e3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:42.201727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f7de4-acfc-48b3-a2da-1bef3e36f9e3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:42.206321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd52aca-5c70-4c35-bfc8-afd0777230f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:42.206918 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd52aca-5c70-4c35-bfc8-afd0777230f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:42.207228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd52aca-5c70-4c35-bfc8-afd0777230f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:42.207501 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd52aca-5c70-4c35-bfc8-afd0777230f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:42.208131 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd52aca-5c70-4c35-bfc8-afd0777230f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:42.209705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd52aca-5c70-4c35-bfc8-afd0777230f5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:42.221783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1419/2868] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:20:42.226861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a1f03-cef5-462a-b89f-adf3d9632da3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:42.237672 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a1f03-cef5-462a-b89f-adf3d9632da3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:42.242665 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae379da-cb54-4283-8792-33b16d14983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:42.243537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae379da-cb54-4283-8792-33b16d14983d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:42.243808 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae379da-cb54-4283-8792-33b16d14983d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:42.244076 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae379da-cb54-4283-8792-33b16d14983d admin admin] PATH_INFO: `/v3/projects/b993aaf98c0141d7a8a65c20ad23f7cb/users/11c00a31393d4749a959659bfca25e9a/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:42.261923 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae379da-cb54-4283-8792-33b16d14983d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b993aaf98c0141d7a8a65c20ad23f7cb, user_id=11c00a31393d4749a959659bfca25e9a, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:42.264902 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae379da-cb54-4283-8792-33b16d14983d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:42.280950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1450/2869] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:20:42 2022] PUT /identity/v3/projects/b993aaf98c0141d7a8a65c20ad23f7cb/users/11c00a31393d4749a959659bfca25e9a/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:42.293313 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe54ae6-6952-4001-87b6-f42e64a857be None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:42.293602 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe54ae6-6952-4001-87b6-f42e64a857be None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:42.293858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe54ae6-6952-4001-87b6-f42e64a857be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:42.302869 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:42.327749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bfe54ae6-6952-4001-87b6-f42e64a857be None None] MFA Rules not processed for user `11c00a31393d4749a959659bfca25e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:42.328955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bfe54ae6-6952-4001-87b6-f42e64a857be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:42.393541 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1420/2870] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:42.404766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae379da-cb54-4283-8792-33b16d14983d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:42.406626 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1ae379da-cb54-4283-8792-33b16d14983d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:42.435783 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:42.468448 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae379da-cb54-4283-8792-33b16d14983d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:42.472404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4edb22-2b14-418a-a37a-bdd2e37134cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:42.473002 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4edb22-2b14-418a-a37a-bdd2e37134cf service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:42.473280 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4edb22-2b14-418a-a37a-bdd2e37134cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:42.473548 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4edb22-2b14-418a-a37a-bdd2e37134cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:42.481795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4edb22-2b14-418a-a37a-bdd2e37134cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:42.483226 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4edb22-2b14-418a-a37a-bdd2e37134cf service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:42.491961 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1451/2871] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:20:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:20:42.511720 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe54ae6-6952-4001-87b6-f42e64a857be None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:42.575096 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe54ae6-6952-4001-87b6-f42e64a857be None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:42.579748 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:42.580390 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:42.580713 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:42.580970 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] PATH_INFO: `/v3/users/11c00a31393d4749a959659bfca25e9a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:42.582103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11c00a31393d4749a959659bfca25e9a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:42.584056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:42.630208 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11c00a31393d4749a959659bfca25e9a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:42.642529 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11c00a31393d4749a959659bfca25e9a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:42.650086 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11c00a31393d4749a959659bfca25e9a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:42.650842 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11c00a31393d4749a959659bfca25e9a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:42.658268 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11c00a31393d4749a959659bfca25e9a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:42.664352 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1421/2872] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:20:42 2022] DELETE /identity/v3/users/11c00a31393d4749a959659bfca25e9a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:42.669400 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d992d54-a231-402e-9841-aac5dec81a6a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:42.669673 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d992d54-a231-402e-9841-aac5dec81a6a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:42.669953 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d992d54-a231-402e-9841-aac5dec81a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:42.689705 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-4d992d54-a231-402e-9841-aac5dec81a6a None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:42.690908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4d992d54-a231-402e-9841-aac5dec81a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:42.749537 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1452/2873] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:42.759386 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:42.760971 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:42.774827 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:42.822111 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee912b15-fc21-460a-aeca-74a184b87e39 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:42.825775 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3782b-e25e-404e-a0ca-50eba1a8df2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:42.826378 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3782b-e25e-404e-a0ca-50eba1a8df2f service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:42.826657 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3782b-e25e-404e-a0ca-50eba1a8df2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:42.826931 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3782b-e25e-404e-a0ca-50eba1a8df2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:42.835833 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3782b-e25e-404e-a0ca-50eba1a8df2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:42.837260 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3782b-e25e-404e-a0ca-50eba1a8df2f service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:42.847070 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1422/2874] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:20:43.358766 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d992d54-a231-402e-9841-aac5dec81a6a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:43.360210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4d992d54-a231-402e-9841-aac5dec81a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:43.372270 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d992d54-a231-402e-9841-aac5dec81a6a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:43.376895 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:43.377523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:43.378007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:43.378249 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] PATH_INFO: `/v3/projects/b993aaf98c0141d7a8a65c20ad23f7cb` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:43.379422 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b993aaf98c0141d7a8a65c20ad23f7cb)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:43.381403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:43.428403 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b993aaf98c0141d7a8a65c20ad23f7cb', 'request_id': 'req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:43.434569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b993aaf98c0141d7a8a65c20ad23f7cb', 'request_id': 'req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:43.440396 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b993aaf98c0141d7a8a65c20ad23f7cb', 'request_id': 'req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:43.453119 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] The token cache is being invalidate because project b993aaf98c0141d7a8a65c20ad23f7cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:20:43.453413 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:43.454048 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1d8a96de-9e04-4058-8f1c-0e30e44ce9c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:20:43.455563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1453/2875] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:20:43 2022] DELETE /identity/v3/projects/b993aaf98c0141d7a8a65c20ad23f7cb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:20:43.460468 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1001e28-1688-4d18-924c-402f230cbe0d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:43.460752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1001e28-1688-4d18-924c-402f230cbe0d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:43.461026 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1001e28-1688-4d18-924c-402f230cbe0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:43.480736 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d1001e28-1688-4d18-924c-402f230cbe0d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:43.481932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d1001e28-1688-4d18-924c-402f230cbe0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:43.541447 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1423/2876] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:43.549756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bd8d0-63ab-4f0c-bbb3-2ef62dacc3d6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:43.550094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bd8d0-63ab-4f0c-bbb3-2ef62dacc3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:43.550386 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bd8d0-63ab-4f0c-bbb3-2ef62dacc3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:43.570196 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-708bd8d0-63ab-4f0c-bbb3-2ef62dacc3d6 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:43.571417 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-708bd8d0-63ab-4f0c-bbb3-2ef62dacc3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:43.579640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1454/2877] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:43.583915 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1001e28-1688-4d18-924c-402f230cbe0d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:43.594342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1001e28-1688-4d18-924c-402f230cbe0d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:43.599628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3454bd-1585-4d83-a0c6-c36f3b61a312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:43.600262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3454bd-1585-4d83-a0c6-c36f3b61a312 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:43.600655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3454bd-1585-4d83-a0c6-c36f3b61a312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:43.601125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3454bd-1585-4d83-a0c6-c36f3b61a312 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:43.601729 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3454bd-1585-4d83-a0c6-c36f3b61a312 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:20:43.602032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3454bd-1585-4d83-a0c6-c36f3b61a312 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:43.603676 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3454bd-1585-4d83-a0c6-c36f3b61a312 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:43.615525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1424/2878] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:20:43.620074 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bd8d0-63ab-4f0c-bbb3-2ef62dacc3d6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:43.629352 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bd8d0-63ab-4f0c-bbb3-2ef62dacc3d6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:43.634138 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc31d2-60c1-48c5-a250-2063859f6622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:43.634734 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc31d2-60c1-48c5-a250-2063859f6622 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:43.635051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc31d2-60c1-48c5-a250-2063859f6622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:43.635328 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc31d2-60c1-48c5-a250-2063859f6622 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:43.636461 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc31d2-60c1-48c5-a250-2063859f6622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:43.638340 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc31d2-60c1-48c5-a250-2063859f6622 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:43.665829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1455/2879] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:20:43 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:20:43.669803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3454bd-1585-4d83-a0c6-c36f3b61a312 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:43.679881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3454bd-1585-4d83-a0c6-c36f3b61a312 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:43.734180 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34540d-5925-46f8-bb90-ba09e0845569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:43.734783 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34540d-5925-46f8-bb90-ba09e0845569 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:43.735078 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34540d-5925-46f8-bb90-ba09e0845569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:43.735349 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34540d-5925-46f8-bb90-ba09e0845569 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:43.735974 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34540d-5925-46f8-bb90-ba09e0845569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:43.737896 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34540d-5925-46f8-bb90-ba09e0845569 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:43.759482 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1425/2880] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:20:43 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:20:43.764152 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc31d2-60c1-48c5-a250-2063859f6622 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:43.774279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc31d2-60c1-48c5-a250-2063859f6622 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:43.779471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d8c1d-77ea-488c-8b84-e1e580005801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:43.780064 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d8c1d-77ea-488c-8b84-e1e580005801 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:43.780344 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d8c1d-77ea-488c-8b84-e1e580005801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:43.780613 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d8c1d-77ea-488c-8b84-e1e580005801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:43.781252 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d8c1d-77ea-488c-8b84-e1e580005801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:43.782820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d8c1d-77ea-488c-8b84-e1e580005801 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:43.792670 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1456/2881] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:20:43.796655 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34540d-5925-46f8-bb90-ba09e0845569 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:43.806733 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34540d-5925-46f8-bb90-ba09e0845569 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:43.811749 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e036ff5-408c-43e0-8864-0bbedf54e26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:43.812330 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e036ff5-408c-43e0-8864-0bbedf54e26e admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:43.812608 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e036ff5-408c-43e0-8864-0bbedf54e26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:43.812875 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e036ff5-408c-43e0-8864-0bbedf54e26e admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:43.813527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e036ff5-408c-43e0-8864-0bbedf54e26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:43.815132 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e036ff5-408c-43e0-8864-0bbedf54e26e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:43.825973 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1426/2882] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:20:43.830016 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d8c1d-77ea-488c-8b84-e1e580005801 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:43.840231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d8c1d-77ea-488c-8b84-e1e580005801 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:43.845129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78082811-d398-4242-a009-4c5785701221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:43.845770 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78082811-d398-4242-a009-4c5785701221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:43.846054 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78082811-d398-4242-a009-4c5785701221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:43.846332 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78082811-d398-4242-a009-4c5785701221 admin admin] PATH_INFO: `/v3/projects/051bad96e24d43d1a4841ed285f0ce62/users/5d55e206d114451abc2e1a241b412c82/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:43.855740 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:43.863176 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78082811-d398-4242-a009-4c5785701221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=051bad96e24d43d1a4841ed285f0ce62, user_id=5d55e206d114451abc2e1a241b412c82, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:43.866488 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78082811-d398-4242-a009-4c5785701221 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:43.880170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1457/2883] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:20:43 2022] PUT /identity/v3/projects/051bad96e24d43d1a4841ed285f0ce62/users/5d55e206d114451abc2e1a241b412c82/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:20:43.891009 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f7f58-c2e0-493e-b6b2-4c60bf3b66e0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:43.891629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f7f58-c2e0-493e-b6b2-4c60bf3b66e0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:43.891629 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f7f58-c2e0-493e-b6b2-4c60bf3b66e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:43.918550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0b3f7f58-c2e0-493e-b6b2-4c60bf3b66e0 None None] MFA Rules not processed for user `5d55e206d114451abc2e1a241b412c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:20:43.919858 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0b3f7f58-c2e0-493e-b6b2-4c60bf3b66e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:43.928126 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:43.984395 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1427/2884] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:20:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:20:43.995849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78082811-d398-4242-a009-4c5785701221 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:20:43.997498 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-78082811-d398-4242-a009-4c5785701221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:20:44.056019 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78082811-d398-4242-a009-4c5785701221 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:20:44.060306 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241271f-afc0-430c-b2aa-128655306685 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6103e9a612c84a9ea7d21cc5128fa40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:20:44.060908 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241271f-afc0-430c-b2aa-128655306685 service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:20:44.061188 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241271f-afc0-430c-b2aa-128655306685 service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:20:44.061457 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241271f-afc0-430c-b2aa-128655306685 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:20:44.069857 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241271f-afc0-430c-b2aa-128655306685 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:20:44.071321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241271f-afc0-430c-b2aa-128655306685 service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:20:44.080797 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1458/2885] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:20:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:20:53.981474 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=23 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:20:54.076795 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:05.942934 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f7f58-c2e0-493e-b6b2-4c60bf3b66e0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:06.006460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f7f58-c2e0-493e-b6b2-4c60bf3b66e0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:06.011550 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:06.012182 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:06.012461 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:06.012734 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] PATH_INFO: `/v3/users/5d55e206d114451abc2e1a241b412c82` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:06.013860 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d55e206d114451abc2e1a241b412c82)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:06.015803 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:06.064891 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d55e206d114451abc2e1a241b412c82'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:06.077846 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d55e206d114451abc2e1a241b412c82'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:06.084955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d55e206d114451abc2e1a241b412c82'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:06.085678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d55e206d114451abc2e1a241b412c82'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:06.095198 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d55e206d114451abc2e1a241b412c82'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:06.101567 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1428/2886] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:21:05 2022] DELETE /identity/v3/users/5d55e206d114451abc2e1a241b412c82 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:21:06.106387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66edbf3-202a-4f07-a929-e2b21029f657 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:06.106930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66edbf3-202a-4f07-a929-e2b21029f657 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:06.106930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66edbf3-202a-4f07-a929-e2b21029f657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:06.125975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b66edbf3-202a-4f07-a929-e2b21029f657 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:06.127232 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b66edbf3-202a-4f07-a929-e2b21029f657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:06.187923 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1459/2887] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:21:06.198647 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:06.200138 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:06.260664 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160bc5b-31ca-430e-aaf8-ec36cadcc69c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:06.264445 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14652608-7dca-4bcf-b83a-bfcf2fc682ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:06.265056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652608-7dca-4bcf-b83a-bfcf2fc682ba service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:06.265334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652608-7dca-4bcf-b83a-bfcf2fc682ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:06.265605 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652608-7dca-4bcf-b83a-bfcf2fc682ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:06.277573 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14652608-7dca-4bcf-b83a-bfcf2fc682ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:06.279781 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14652608-7dca-4bcf-b83a-bfcf2fc682ba service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:06.290652 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1429/2888] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:21:06.598664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66edbf3-202a-4f07-a929-e2b21029f657 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:06.600244 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b66edbf3-202a-4f07-a929-e2b21029f657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:06.612667 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66edbf3-202a-4f07-a929-e2b21029f657 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:06.617430 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:06.618088 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:06.618362 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:06.618628 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] PATH_INFO: `/v3/projects/051bad96e24d43d1a4841ed285f0ce62` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:06.620324 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] RBAC: Authorizing `identity:delete_project(project_id=051bad96e24d43d1a4841ed285f0ce62)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:06.622241 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:06.669635 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '051bad96e24d43d1a4841ed285f0ce62', 'request_id': 'req-a237ca88-d917-4ec4-a158-cf4d3a13bcee'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:06.677968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '051bad96e24d43d1a4841ed285f0ce62', 'request_id': 'req-a237ca88-d917-4ec4-a158-cf4d3a13bcee'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:06.684360 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '051bad96e24d43d1a4841ed285f0ce62', 'request_id': 'req-a237ca88-d917-4ec4-a158-cf4d3a13bcee'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:06.697245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] The token cache is being invalidate because project 051bad96e24d43d1a4841ed285f0ce62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:21:06.697462 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:06.698118 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a237ca88-d917-4ec4-a158-cf4d3a13bcee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:06.699640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1460/2889] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:21:06 2022] DELETE /identity/v3/projects/051bad96e24d43d1a4841ed285f0ce62 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:21:06.704871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee6a45-f899-4fbe-9b21-3efaa8c0da3b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:06.705152 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee6a45-f899-4fbe-9b21-3efaa8c0da3b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:06.705434 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee6a45-f899-4fbe-9b21-3efaa8c0da3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:06.725807 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-46ee6a45-f899-4fbe-9b21-3efaa8c0da3b None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:06.727052 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-46ee6a45-f899-4fbe-9b21-3efaa8c0da3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:06.747103 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:06.800348 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1430/2890] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:21:06.868326 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0196d-6152-4ee8-ba93-0a25a246c90c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:06.868648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0196d-6152-4ee8-ba93-0a25a246c90c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:06.868939 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0196d-6152-4ee8-ba93-0a25a246c90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:06.892432 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-74c0196d-6152-4ee8-ba93-0a25a246c90c None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:06.893689 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-74c0196d-6152-4ee8-ba93-0a25a246c90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:06.902090 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1461/2891] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:21:06.906618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee6a45-f899-4fbe-9b21-3efaa8c0da3b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:06.917183 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee6a45-f899-4fbe-9b21-3efaa8c0da3b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:06.921845 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdf597-7b0b-4e1a-8e59-fd0d74e2df80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:06.922453 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdf597-7b0b-4e1a-8e59-fd0d74e2df80 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:06.922727 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdf597-7b0b-4e1a-8e59-fd0d74e2df80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:06.923032 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdf597-7b0b-4e1a-8e59-fd0d74e2df80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:06.923688 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bdf597-7b0b-4e1a-8e59-fd0d74e2df80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:21:06.923984 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bdf597-7b0b-4e1a-8e59-fd0d74e2df80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:06.925570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bdf597-7b0b-4e1a-8e59-fd0d74e2df80 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:06.938654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1431/2892] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:21:06.943329 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0196d-6152-4ee8-ba93-0a25a246c90c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:06.953756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0196d-6152-4ee8-ba93-0a25a246c90c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:06.958721 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e92a92-da0a-496b-87cd-6a3afe930579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:06.959377 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e92a92-da0a-496b-87cd-6a3afe930579 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:06.959651 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e92a92-da0a-496b-87cd-6a3afe930579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:06.959922 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e92a92-da0a-496b-87cd-6a3afe930579 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:06.960569 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e92a92-da0a-496b-87cd-6a3afe930579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:06.962456 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e92a92-da0a-496b-87cd-6a3afe930579 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:06.993584 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1462/2893] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:21:06 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 17:21:06.998520 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdf597-7b0b-4e1a-8e59-fd0d74e2df80 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:07.009234 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdf597-7b0b-4e1a-8e59-fd0d74e2df80 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:07.070354 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dae1f7-7854-4298-955d-497eee42f7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:07.071001 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dae1f7-7854-4298-955d-497eee42f7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:07.071285 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dae1f7-7854-4298-955d-497eee42f7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:07.071553 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dae1f7-7854-4298-955d-497eee42f7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:07.072182 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dae1f7-7854-4298-955d-497eee42f7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:07.074098 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dae1f7-7854-4298-955d-497eee42f7c2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:07.098010 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1432/2894] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:21:06 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:21:07.102516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e92a92-da0a-496b-87cd-6a3afe930579 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:07.113789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e92a92-da0a-496b-87cd-6a3afe930579 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:07.118383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89997adc-acf6-4b75-9baa-927216cb1dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:07.119012 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997adc-acf6-4b75-9baa-927216cb1dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:07.119297 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997adc-acf6-4b75-9baa-927216cb1dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:07.119581 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997adc-acf6-4b75-9baa-927216cb1dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:07.120213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89997adc-acf6-4b75-9baa-927216cb1dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:07.121794 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89997adc-acf6-4b75-9baa-927216cb1dd5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:07.131935 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1463/2895] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:21:07.136132 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dae1f7-7854-4298-955d-497eee42f7c2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:07.146790 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dae1f7-7854-4298-955d-497eee42f7c2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:07.152356 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fde55-9ffe-4a74-823b-fb26680e388d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:07.152929 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fde55-9ffe-4a74-823b-fb26680e388d admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:07.153194 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fde55-9ffe-4a74-823b-fb26680e388d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:07.153463 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fde55-9ffe-4a74-823b-fb26680e388d admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:07.154104 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fde55-9ffe-4a74-823b-fb26680e388d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:07.155735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fde55-9ffe-4a74-823b-fb26680e388d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:07.167480 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1433/2896] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:21:07.171777 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89997adc-acf6-4b75-9baa-927216cb1dd5 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:07.182215 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89997adc-acf6-4b75-9baa-927216cb1dd5 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:07.186833 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19cce9-89d5-4ea2-9b7c-f356ca128d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:07.187578 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19cce9-89d5-4ea2-9b7c-f356ca128d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:07.187855 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19cce9-89d5-4ea2-9b7c-f356ca128d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:07.188124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19cce9-89d5-4ea2-9b7c-f356ca128d74 admin admin] PATH_INFO: `/v3/projects/09331f6d863d4038adb693dd60ba266c/users/e96e6ad4fa5949cea875ce756c3ca22f/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:07.202914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19cce9-89d5-4ea2-9b7c-f356ca128d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09331f6d863d4038adb693dd60ba266c, user_id=e96e6ad4fa5949cea875ce756c3ca22f, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:07.205640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19cce9-89d5-4ea2-9b7c-f356ca128d74 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:07.222575 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1464/2897] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:21:07 2022] PUT /identity/v3/projects/09331f6d863d4038adb693dd60ba266c/users/e96e6ad4fa5949cea875ce756c3ca22f/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:21:07.233856 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1788b5-3de2-49c6-afb6-e71fad74ff46 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:07.234139 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1788b5-3de2-49c6-afb6-e71fad74ff46 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:07.234464 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1788b5-3de2-49c6-afb6-e71fad74ff46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:07.263878 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4e1788b5-3de2-49c6-afb6-e71fad74ff46 None None] MFA Rules not processed for user `e96e6ad4fa5949cea875ce756c3ca22f`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:07.265115 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4e1788b5-3de2-49c6-afb6-e71fad74ff46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:07.334575 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1434/2898] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:21:07.370295 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19cce9-89d5-4ea2-9b7c-f356ca128d74 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:07.372027 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1b19cce9-89d5-4ea2-9b7c-f356ca128d74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:07.393951 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:07.432592 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19cce9-89d5-4ea2-9b7c-f356ca128d74 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:07.436563 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538032b-3c55-4c57-9ce2-a9e6fccd0d3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:07.437174 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538032b-3c55-4c57-9ce2-a9e6fccd0d3d service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:07.437455 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538032b-3c55-4c57-9ce2-a9e6fccd0d3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:07.437730 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538032b-3c55-4c57-9ce2-a9e6fccd0d3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:07.446769 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538032b-3c55-4c57-9ce2-a9e6fccd0d3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:07.448722 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538032b-3c55-4c57-9ce2-a9e6fccd0d3d service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:07.457862 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1465/2899] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:21:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:21:07.473854 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1788b5-3de2-49c6-afb6-e71fad74ff46 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:07.493367 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:07.544226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1788b5-3de2-49c6-afb6-e71fad74ff46 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:07.550394 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:07.551069 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:07.551628 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:07.551922 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] PATH_INFO: `/v3/users/e96e6ad4fa5949cea875ce756c3ca22f` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:07.553063 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e96e6ad4fa5949cea875ce756c3ca22f)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:07.555023 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:07.606618 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e96e6ad4fa5949cea875ce756c3ca22f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:07.618844 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e96e6ad4fa5949cea875ce756c3ca22f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:07.625735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e96e6ad4fa5949cea875ce756c3ca22f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:07.626430 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e96e6ad4fa5949cea875ce756c3ca22f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:07.635460 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e96e6ad4fa5949cea875ce756c3ca22f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:07.642007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1435/2900] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:21:07 2022] DELETE /identity/v3/users/e96e6ad4fa5949cea875ce756c3ca22f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:21:07.646853 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb172d2-a0ab-4dd2-80b5-5352f4b0d87f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:07.647186 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb172d2-a0ab-4dd2-80b5-5352f4b0d87f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:07.647467 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb172d2-a0ab-4dd2-80b5-5352f4b0d87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:07.667231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-fcb172d2-a0ab-4dd2-80b5-5352f4b0d87f None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:07.668449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fcb172d2-a0ab-4dd2-80b5-5352f4b0d87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:07.729922 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1466/2901] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:21:07.739650 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:07.741190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:07.800789 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acd9fc-6b1f-4895-8d54-8935c663a924 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:07.804741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9a73e-cfbf-43ec-aedf-8cef6b1f48ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:07.805345 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a73e-cfbf-43ec-aedf-8cef6b1f48ea service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:07.805626 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a73e-cfbf-43ec-aedf-8cef6b1f48ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:07.805901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a73e-cfbf-43ec-aedf-8cef6b1f48ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:07.815470 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9a73e-cfbf-43ec-aedf-8cef6b1f48ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:07.816905 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9a73e-cfbf-43ec-aedf-8cef6b1f48ea service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:07.827337 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1436/2902] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 17:21:08.107449 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb172d2-a0ab-4dd2-80b5-5352f4b0d87f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:08.109142 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fcb172d2-a0ab-4dd2-80b5-5352f4b0d87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:08.122525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb172d2-a0ab-4dd2-80b5-5352f4b0d87f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:08.131228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:08.132466 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:08.133337 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:08.134069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] PATH_INFO: `/v3/projects/09331f6d863d4038adb693dd60ba266c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:08.142849 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09331f6d863d4038adb693dd60ba266c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:08.148130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:08.202476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09331f6d863d4038adb693dd60ba266c', 'request_id': 'req-f1041aff-cd90-4b62-9fd2-071b9008f87e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:08.208930 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09331f6d863d4038adb693dd60ba266c', 'request_id': 'req-f1041aff-cd90-4b62-9fd2-071b9008f87e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:08.215648 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09331f6d863d4038adb693dd60ba266c', 'request_id': 'req-f1041aff-cd90-4b62-9fd2-071b9008f87e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:08.230785 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] The token cache is being invalidate because project 09331f6d863d4038adb693dd60ba266c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:21:08.231129 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:08.231671 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f1041aff-cd90-4b62-9fd2-071b9008f87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:08.233285 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1467/2903] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:21:08 2022] DELETE /identity/v3/projects/09331f6d863d4038adb693dd60ba266c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:21:08.238241 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241be67-5191-463f-807f-c61ac1ae0c2d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:08.239105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241be67-5191-463f-807f-c61ac1ae0c2d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:08.239105 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241be67-5191-463f-807f-c61ac1ae0c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:08.260705 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5241be67-5191-463f-807f-c61ac1ae0c2d None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:08.261950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5241be67-5191-463f-807f-c61ac1ae0c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:08.298543 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:08.327678 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1437/2904] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:21:08.335647 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea031fc1-4790-4dd5-a0a3-ed533fbb18ef None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:08.336049 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea031fc1-4790-4dd5-a0a3-ed533fbb18ef None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:08.336336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea031fc1-4790-4dd5-a0a3-ed533fbb18ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:08.359474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ea031fc1-4790-4dd5-a0a3-ed533fbb18ef None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:08.360719 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ea031fc1-4790-4dd5-a0a3-ed533fbb18ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:08.369261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1468/2905] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:21:08.374724 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241be67-5191-463f-807f-c61ac1ae0c2d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:08.385525 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241be67-5191-463f-807f-c61ac1ae0c2d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:08.390876 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81d02e-988b-4716-acea-21863369a3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:08.391551 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81d02e-988b-4716-acea-21863369a3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:08.391835 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81d02e-988b-4716-acea-21863369a3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:08.392110 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81d02e-988b-4716-acea-21863369a3a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:08.392800 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81d02e-988b-4716-acea-21863369a3a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:21:08.393106 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81d02e-988b-4716-acea-21863369a3a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:08.394741 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81d02e-988b-4716-acea-21863369a3a4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:08.407818 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1438/2906] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:21:08.412397 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea031fc1-4790-4dd5-a0a3-ed533fbb18ef None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:08.422629 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea031fc1-4790-4dd5-a0a3-ed533fbb18ef None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:08.427757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac23d6-bb2f-4c6e-b2a9-ab6ba57e7fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:08.428376 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac23d6-bb2f-4c6e-b2a9-ab6ba57e7fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:08.428657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac23d6-bb2f-4c6e-b2a9-ab6ba57e7fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:08.428962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac23d6-bb2f-4c6e-b2a9-ab6ba57e7fd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:08.429646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac23d6-bb2f-4c6e-b2a9-ab6ba57e7fd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:08.431657 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac23d6-bb2f-4c6e-b2a9-ab6ba57e7fd3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:08.464825 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1469/2907] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:21:08 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:21:08.469185 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81d02e-988b-4716-acea-21863369a3a4 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:08.480039 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81d02e-988b-4716-acea-21863369a3a4 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:08.539157 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01b247-c845-4c6c-b71a-5cb0311d0c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:08.539682 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01b247-c845-4c6c-b71a-5cb0311d0c3a admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:08.539963 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01b247-c845-4c6c-b71a-5cb0311d0c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:08.540236 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01b247-c845-4c6c-b71a-5cb0311d0c3a admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:08.540871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01b247-c845-4c6c-b71a-5cb0311d0c3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:08.543389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01b247-c845-4c6c-b71a-5cb0311d0c3a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:08.564534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1439/2908] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:21:08 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:21:08.568861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac23d6-bb2f-4c6e-b2a9-ab6ba57e7fd3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:08.579202 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac23d6-bb2f-4c6e-b2a9-ab6ba57e7fd3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:08.583852 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d5943-1453-4cd6-9db1-e3b4e755997a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:08.584452 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d5943-1453-4cd6-9db1-e3b4e755997a admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:08.584714 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d5943-1453-4cd6-9db1-e3b4e755997a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:08.584988 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d5943-1453-4cd6-9db1-e3b4e755997a admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:08.585621 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d5943-1453-4cd6-9db1-e3b4e755997a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:08.587231 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d5943-1453-4cd6-9db1-e3b4e755997a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:08.598201 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1470/2909] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:21:08.602421 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01b247-c845-4c6c-b71a-5cb0311d0c3a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:08.613955 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01b247-c845-4c6c-b71a-5cb0311d0c3a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:08.618563 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391c4f6-da35-4e55-bcb6-4a7207e14428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:08.619195 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c4f6-da35-4e55-bcb6-4a7207e14428 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:08.619500 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c4f6-da35-4e55-bcb6-4a7207e14428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:08.619773 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c4f6-da35-4e55-bcb6-4a7207e14428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:08.620407 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391c4f6-da35-4e55-bcb6-4a7207e14428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:08.622015 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391c4f6-da35-4e55-bcb6-4a7207e14428 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:08.633033 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1440/2910] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 17:21:08.637476 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d5943-1453-4cd6-9db1-e3b4e755997a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:08.647820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d5943-1453-4cd6-9db1-e3b4e755997a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:08.653523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9ff0d-bd9a-4535-8014-51f322d70e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:08.654171 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9ff0d-bd9a-4535-8014-51f322d70e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:08.654482 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9ff0d-bd9a-4535-8014-51f322d70e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:08.654757 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9ff0d-bd9a-4535-8014-51f322d70e6f admin admin] PATH_INFO: `/v3/projects/80d8064f1a904b6e8b0b4db7ce39e837/users/d8291d514bd847b1a55f5c225029c295/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:08.657228 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:08.673636 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9ff0d-bd9a-4535-8014-51f322d70e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d8064f1a904b6e8b0b4db7ce39e837, user_id=d8291d514bd847b1a55f5c225029c295, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:08.676688 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9ff0d-bd9a-4535-8014-51f322d70e6f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:08.692383 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1471/2911] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:21:08 2022] PUT /identity/v3/projects/80d8064f1a904b6e8b0b4db7ce39e837/users/d8291d514bd847b1a55f5c225029c295/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:21:08.704786 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d18a2-a728-4ceb-87de-88b46890eba4 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:08.705104 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d18a2-a728-4ceb-87de-88b46890eba4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:08.705401 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d18a2-a728-4ceb-87de-88b46890eba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:08.734737 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3e8d18a2-a728-4ceb-87de-88b46890eba4 None None] MFA Rules not processed for user `d8291d514bd847b1a55f5c225029c295`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:08.735988 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3e8d18a2-a728-4ceb-87de-88b46890eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:08.810042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1441/2912] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:21:08.821108 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9ff0d-bd9a-4535-8014-51f322d70e6f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:08.822793 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-95b9ff0d-bd9a-4535-8014-51f322d70e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:08.880880 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9ff0d-bd9a-4535-8014-51f322d70e6f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:08.884673 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c817ef0-2548-4d8a-92f0-f1a8e18be062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae1b8a51c9fa4f2594bdd79b58962b0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:08.885289 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c817ef0-2548-4d8a-92f0-f1a8e18be062 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:08.885604 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c817ef0-2548-4d8a-92f0-f1a8e18be062 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:08.885874 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c817ef0-2548-4d8a-92f0-f1a8e18be062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:08.895123 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c817ef0-2548-4d8a-92f0-f1a8e18be062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:08.896591 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c817ef0-2548-4d8a-92f0-f1a8e18be062 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:08.906321 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1472/2913] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:21:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:21:15.291994 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d18a2-a728-4ceb-87de-88b46890eba4 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:15.293765 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3e8d18a2-a728-4ceb-87de-88b46890eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:15.335389 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:15.366532 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d18a2-a728-4ceb-87de-88b46890eba4 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:15.371982 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0da266-39a9-4c58-8bbc-abb04eab5497 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:15.372611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0da266-39a9-4c58-8bbc-abb04eab5497 service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:15.372890 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0da266-39a9-4c58-8bbc-abb04eab5497 service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:15.373165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0da266-39a9-4c58-8bbc-abb04eab5497 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:15.374925 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6e0da266-39a9-4c58-8bbc-abb04eab5497 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:15.387302 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0da266-39a9-4c58-8bbc-abb04eab5497 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:15.388893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0da266-39a9-4c58-8bbc-abb04eab5497 service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:15.399332 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1442/2914] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:21:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:21:18.902147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:25.394778 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=16 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:30.318206 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c817ef0-2548-4d8a-92f0-f1a8e18be062 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:30.320843 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2c817ef0-2548-4d8a-92f0-f1a8e18be062 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:30.417031 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c817ef0-2548-4d8a-92f0-f1a8e18be062 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:30.424147 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4fd8a-c5e6-4571-a015-cab17e1ccd96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6c1ce47d8704111988a78e625fd968c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:30.424782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4fd8a-c5e6-4571-a015-cab17e1ccd96 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:30.425062 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4fd8a-c5e6-4571-a015-cab17e1ccd96 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:30.425342 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4fd8a-c5e6-4571-a015-cab17e1ccd96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:30.427556 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f9c4fd8a-c5e6-4571-a015-cab17e1ccd96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:30.443471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4fd8a-c5e6-4571-a015-cab17e1ccd96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:30.445133 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4fd8a-c5e6-4571-a015-cab17e1ccd96 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:30.471990 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1473/2915] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:21:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:21:34.175417 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0da266-39a9-4c58-8bbc-abb04eab5497 service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:34.247276 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0da266-39a9-4c58-8bbc-abb04eab5497 service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:34.251895 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:34.252532 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:34.252815 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:34.253090 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] PATH_INFO: `/v3/users/d8291d514bd847b1a55f5c225029c295` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:34.254228 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8291d514bd847b1a55f5c225029c295)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:34.256201 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:34.307422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8291d514bd847b1a55f5c225029c295'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:34.325802 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8291d514bd847b1a55f5c225029c295'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:34.334136 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8291d514bd847b1a55f5c225029c295'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:34.334880 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8291d514bd847b1a55f5c225029c295'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:34.342631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8291d514bd847b1a55f5c225029c295'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:34.348612 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1443/2916] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:21:34 2022] DELETE /identity/v3/users/d8291d514bd847b1a55f5c225029c295 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:21:34.353290 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dbd86-6120-4df5-849f-9465840cc5c9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:34.353805 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dbd86-6120-4df5-849f-9465840cc5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:34.353937 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dbd86-6120-4df5-849f-9465840cc5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:34.373170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ba4dbd86-6120-4df5-849f-9465840cc5c9 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:34.374357 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ba4dbd86-6120-4df5-849f-9465840cc5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:34.433474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1474/2917] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:21:34.443284 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:34.444670 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:34.500300 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1643f-56c9-4026-9d11-43209a8ad41d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:34.503893 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccfa59-b557-4fe7-a732-85faa50bc13b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:34.504488 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccfa59-b557-4fe7-a732-85faa50bc13b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:34.504788 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccfa59-b557-4fe7-a732-85faa50bc13b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:34.505056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccfa59-b557-4fe7-a732-85faa50bc13b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:34.514073 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccfa59-b557-4fe7-a732-85faa50bc13b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:34.515580 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccfa59-b557-4fe7-a732-85faa50bc13b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:34.525012 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1444/2918] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:21:34.796963 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4dbd86-6120-4df5-849f-9465840cc5c9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:34.798431 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ba4dbd86-6120-4df5-849f-9465840cc5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:34.810320 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4dbd86-6120-4df5-849f-9465840cc5c9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:34.814914 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:34.816474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:34.816772 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:34.817391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] PATH_INFO: `/v3/projects/80d8064f1a904b6e8b0b4db7ce39e837` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:34.818614 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d8064f1a904b6e8b0b4db7ce39e837)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:34.820425 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:34.870395 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d8064f1a904b6e8b0b4db7ce39e837', 'request_id': 'req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:34.876782 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d8064f1a904b6e8b0b4db7ce39e837', 'request_id': 'req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:34.882591 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d8064f1a904b6e8b0b4db7ce39e837', 'request_id': 'req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:34.895203 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] The token cache is being invalidate because project 80d8064f1a904b6e8b0b4db7ce39e837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:21:34.895493 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:34.896124 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-54c98e74-ff4c-4940-8ca6-fd044cb93a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:34.897600 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1475/2919] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:21:34 2022] DELETE /identity/v3/projects/80d8064f1a904b6e8b0b4db7ce39e837 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:21:34.903176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf7fc0-21ad-4091-8e7b-466f175402b3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:34.903466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf7fc0-21ad-4091-8e7b-466f175402b3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:34.903740 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf7fc0-21ad-4091-8e7b-466f175402b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:34.923616 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3dbf7fc0-21ad-4091-8e7b-466f175402b3 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:34.924819 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3dbf7fc0-21ad-4091-8e7b-466f175402b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:34.941654 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:34.988157 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1445/2920] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:21:34.996114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476850b6-6f9c-429f-85e5-353f1d995c25 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:34.996401 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476850b6-6f9c-429f-85e5-353f1d995c25 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:34.996677 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476850b6-6f9c-429f-85e5-353f1d995c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:35.016261 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-476850b6-6f9c-429f-85e5-353f1d995c25 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:35.017471 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-476850b6-6f9c-429f-85e5-353f1d995c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:35.026034 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1476/2921] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 17:21:35.030611 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf7fc0-21ad-4091-8e7b-466f175402b3 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:35.041486 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf7fc0-21ad-4091-8e7b-466f175402b3 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:35.046190 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e392c-e7fd-4a08-9162-2be3e48ca613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:35.046805 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e392c-e7fd-4a08-9162-2be3e48ca613 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:35.047128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e392c-e7fd-4a08-9162-2be3e48ca613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:35.047402 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e392c-e7fd-4a08-9162-2be3e48ca613 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:35.048050 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e392c-e7fd-4a08-9162-2be3e48ca613 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 17:21:35.048353 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e392c-e7fd-4a08-9162-2be3e48ca613 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:35.049959 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e392c-e7fd-4a08-9162-2be3e48ca613 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:35.062466 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1446/2922] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 22 17:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 17:21:35.066901 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476850b6-6f9c-429f-85e5-353f1d995c25 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:35.071391 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:35.078474 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476850b6-6f9c-429f-85e5-353f1d995c25 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:35.084389 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36590769-a3e4-4d01-add0-b5cfbdccaf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:35.085094 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36590769-a3e4-4d01-add0-b5cfbdccaf35 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:35.085443 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36590769-a3e4-4d01-add0-b5cfbdccaf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:35.085886 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36590769-a3e4-4d01-add0-b5cfbdccaf35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:35.086525 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36590769-a3e4-4d01-add0-b5cfbdccaf35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:35.088950 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36590769-a3e4-4d01-add0-b5cfbdccaf35 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:35.122069 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1477/2923] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 22 17:21:35 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:21:35.126504 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e392c-e7fd-4a08-9162-2be3e48ca613 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:35.136519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e392c-e7fd-4a08-9162-2be3e48ca613 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:35.197176 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d0b2b-135e-40c4-8816-1d04d22100c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:35.197932 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d0b2b-135e-40c4-8816-1d04d22100c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:35.198083 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d0b2b-135e-40c4-8816-1d04d22100c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:35.198347 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d0b2b-135e-40c4-8816-1d04d22100c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:35.199007 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d0b2b-135e-40c4-8816-1d04d22100c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:35.200950 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d0b2b-135e-40c4-8816-1d04d22100c2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:35.225635 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1447/2924] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 22 17:21:35 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 17:21:35.230505 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36590769-a3e4-4d01-add0-b5cfbdccaf35 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:35.241309 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36590769-a3e4-4d01-add0-b5cfbdccaf35 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:35.246199 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba4338-fc68-46f5-b081-8283bac9d974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:35.246821 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba4338-fc68-46f5-b081-8283bac9d974 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:35.247134 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba4338-fc68-46f5-b081-8283bac9d974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:35.247404 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba4338-fc68-46f5-b081-8283bac9d974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:35.248050 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba4338-fc68-46f5-b081-8283bac9d974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:35.249640 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba4338-fc68-46f5-b081-8283bac9d974 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:35.261900 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1478/2925] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:21:35.266564 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d0b2b-135e-40c4-8816-1d04d22100c2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:35.279405 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d0b2b-135e-40c4-8816-1d04d22100c2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:35.284752 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39314628-3f74-4ef9-bb81-e13f945d8fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:35.285375 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314628-3f74-4ef9-bb81-e13f945d8fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:35.285653 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314628-3f74-4ef9-bb81-e13f945d8fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:35.285924 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314628-3f74-4ef9-bb81-e13f945d8fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:35.286570 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39314628-3f74-4ef9-bb81-e13f945d8fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:35.288212 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39314628-3f74-4ef9-bb81-e13f945d8fe4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:35.301037 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1448/2926] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 22 17:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 17:21:35.305664 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba4338-fc68-46f5-b081-8283bac9d974 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:35.317789 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba4338-fc68-46f5-b081-8283bac9d974 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:35.323429 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa396ee-7c42-40dd-9116-7dae6cb2ab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:35.324107 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa396ee-7c42-40dd-9116-7dae6cb2ab3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:35.324387 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa396ee-7c42-40dd-9116-7dae6cb2ab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:35.324660 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa396ee-7c42-40dd-9116-7dae6cb2ab3d admin admin] PATH_INFO: `/v3/projects/84d17850960b4ef8b4655554158c3ff9/users/037f63576e404537b4e969487ab47231/roles/34ccf81a9f53404e9edefd846827e5b7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:35.341501 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa396ee-7c42-40dd-9116-7dae6cb2ab3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=84d17850960b4ef8b4655554158c3ff9, user_id=037f63576e404537b4e969487ab47231, role_id=34ccf81a9f53404e9edefd846827e5b7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:35.344130 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa396ee-7c42-40dd-9116-7dae6cb2ab3d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:35.360217 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1479/2927] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 22 17:21:35 2022] PUT /identity/v3/projects/84d17850960b4ef8b4655554158c3ff9/users/037f63576e404537b4e969487ab47231/roles/34ccf81a9f53404e9edefd846827e5b7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:21:35.372135 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf9dc5-d873-4a10-b3c7-d11eb5a553d3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:35.372449 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf9dc5-d873-4a10-b3c7-d11eb5a553d3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:35.372709 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf9dc5-d873-4a10-b3c7-d11eb5a553d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:35.404894 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-31cf9dc5-d873-4a10-b3c7-d11eb5a553d3 None None] MFA Rules not processed for user `037f63576e404537b4e969487ab47231`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:21:35.406125 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-31cf9dc5-d873-4a10-b3c7-d11eb5a553d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:35.483902 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1449/2928] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:21:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:21:35.495687 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa396ee-7c42-40dd-9116-7dae6cb2ab3d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:35.497398 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dfa396ee-7c42-40dd-9116-7dae6cb2ab3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:35.570516 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa396ee-7c42-40dd-9116-7dae6cb2ab3d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:35.574933 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49694d-ab5e-4d61-8aaf-5fb5c845bb26 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6103e9a612c84a9ea7d21cc5128fa40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:35.575574 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49694d-ab5e-4d61-8aaf-5fb5c845bb26 service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:35.575845 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49694d-ab5e-4d61-8aaf-5fb5c845bb26 service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:35.576114 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49694d-ab5e-4d61-8aaf-5fb5c845bb26 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:35.585103 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49694d-ab5e-4d61-8aaf-5fb5c845bb26 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:35.586523 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49694d-ab5e-4d61-8aaf-5fb5c845bb26 service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:35.596170 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1480/2929] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:21:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:21:45.480422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:45.591788 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=49,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:48.192164 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf9dc5-d873-4a10-b3c7-d11eb5a553d3 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:48.193643 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-31cf9dc5-d873-4a10-b3c7-d11eb5a553d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:48.267321 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf9dc5-d873-4a10-b3c7-d11eb5a553d3 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:48.271666 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a9479-236c-44d5-9756-1b1a4dadeab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:48.272396 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a9479-236c-44d5-9756-1b1a4dadeab1 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:48.272784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a9479-236c-44d5-9756-1b1a4dadeab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:48.273163 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a9479-236c-44d5-9756-1b1a4dadeab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:48.275814 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-312a9479-236c-44d5-9756-1b1a4dadeab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:48.346047 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a9479-236c-44d5-9756-1b1a4dadeab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:48.347534 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a9479-236c-44d5-9756-1b1a4dadeab1 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:48.357777 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1450/2930] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:21:51.655750 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49694d-ab5e-4d61-8aaf-5fb5c845bb26 service glance] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:51.657968 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2e49694d-ab5e-4d61-8aaf-5fb5c845bb26 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:51.779732 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49694d-ab5e-4d61-8aaf-5fb5c845bb26 service glance] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:51.785269 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52c1cb-cea4-4d02-83f3-e683479ae151 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:51.785909 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52c1cb-cea4-4d02-83f3-e683479ae151 service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:51.786184 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52c1cb-cea4-4d02-83f3-e683479ae151 service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:51.786458 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52c1cb-cea4-4d02-83f3-e683479ae151 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:51.788322 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1f52c1cb-cea4-4d02-83f3-e683479ae151 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:51.800641 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52c1cb-cea4-4d02-83f3-e683479ae151 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:51.802468 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52c1cb-cea4-4d02-83f3-e683479ae151 service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:51.812865 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1481/2931] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:21:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:21:54.697463 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a9479-236c-44d5-9756-1b1a4dadeab1 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:54.709587 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a9479-236c-44d5-9756-1b1a4dadeab1 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:54.715262 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f090ffd-5463-4969-a3f3-db05fbd7959f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:54.715871 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f090ffd-5463-4969-a3f3-db05fbd7959f service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:54.716148 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f090ffd-5463-4969-a3f3-db05fbd7959f service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:54.716419 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f090ffd-5463-4969-a3f3-db05fbd7959f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:54.718191 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8f090ffd-5463-4969-a3f3-db05fbd7959f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:54.730476 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f090ffd-5463-4969-a3f3-db05fbd7959f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:54.732133 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f090ffd-5463-4969-a3f3-db05fbd7959f service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:54.742793 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1451/2932] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:21:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:21:54.766276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52c1cb-cea4-4d02-83f3-e683479ae151 service placement] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:54.768051 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1f52c1cb-cea4-4d02-83f3-e683479ae151 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:54.787500 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52c1cb-cea4-4d02-83f3-e683479ae151 service placement] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:54.792756 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:54.793472 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:54.793916 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:54.794350 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] PATH_INFO: `/v3/users/c2060b33d0a24e01a96ab3fb02b4b045` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:54.810861 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2060b33d0a24e01a96ab3fb02b4b045)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:54.812842 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:54.866618 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2060b33d0a24e01a96ab3fb02b4b045'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:54.880065 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2060b33d0a24e01a96ab3fb02b4b045'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:54.887646 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2060b33d0a24e01a96ab3fb02b4b045'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:54.895336 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2060b33d0a24e01a96ab3fb02b4b045'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:54.896024 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2060b33d0a24e01a96ab3fb02b4b045'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:54.902885 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1482/2933] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:21:54 2022] DELETE /identity/v3/users/c2060b33d0a24e01a96ab3fb02b4b045 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:21:55.080130 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f090ffd-5463-4969-a3f3-db05fbd7959f service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:55.091545 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f090ffd-5463-4969-a3f3-db05fbd7959f service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:55.151901 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:55.152438 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:55.152709 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:55.152983 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] PATH_INFO: `/v3/projects/5fca6eabbca24bc09aee7e259ca82562` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:55.163145 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fca6eabbca24bc09aee7e259ca82562)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:55.165042 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:55.199703 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:55.213697 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fca6eabbca24bc09aee7e259ca82562', 'request_id': 'req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.219735 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fca6eabbca24bc09aee7e259ca82562', 'request_id': 'req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.228648 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fca6eabbca24bc09aee7e259ca82562', 'request_id': 'req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.245242 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] The token cache is being invalidate because project 5fca6eabbca24bc09aee7e259ca82562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:21:55.245596 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.246519 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.248327 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1452/2934] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:21:55 2022] DELETE /identity/v3/projects/5fca6eabbca24bc09aee7e259ca82562 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 17:21:55.252245 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:55.253720 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:55.267300 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457b3c7-5871-4436-8f22-0e617e13fd0b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:55.319078 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:21:55.331463 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:55.332109 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:55.332388 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:55.332661 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] PATH_INFO: `/v3/users/da9803645097465887035bf5fe7de010` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:55.356007 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da9803645097465887035bf5fe7de010)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:55.357962 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:55.409650 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da9803645097465887035bf5fe7de010'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.422554 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da9803645097465887035bf5fe7de010'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.429728 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da9803645097465887035bf5fe7de010'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.438213 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da9803645097465887035bf5fe7de010'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.438837 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da9803645097465887035bf5fe7de010'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.445369 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1483/2935] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:21:55 2022] DELETE /identity/v3/users/da9803645097465887035bf5fe7de010 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:21:55.619397 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:21:55.620868 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:21:55.634359 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f305c-5d56-46a2-aaae-a79cd8d38cb2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:21:55.694481 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:21:55.695143 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:21:55.695422 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:21:55.695691 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] PATH_INFO: `/v3/projects/ec3eb12cabb04fd8bba960c1432f65e1` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:21:55.706188 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec3eb12cabb04fd8bba960c1432f65e1)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:21:55.708140 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:21:55.755493 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec3eb12cabb04fd8bba960c1432f65e1', 'request_id': 'req-cf256d34-3d9e-43e0-84fb-db32e2d37b33'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.760975 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec3eb12cabb04fd8bba960c1432f65e1', 'request_id': 'req-cf256d34-3d9e-43e0-84fb-db32e2d37b33'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.768342 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec3eb12cabb04fd8bba960c1432f65e1', 'request_id': 'req-cf256d34-3d9e-43e0-84fb-db32e2d37b33'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.781792 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] The token cache is being invalidate because project ec3eb12cabb04fd8bba960c1432f65e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:21:55.782093 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.782680 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-cf256d34-3d9e-43e0-84fb-db32e2d37b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:21:55.784266 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1453/2936] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:21:55 2022] DELETE /identity/v3/projects/ec3eb12cabb04fd8bba960c1432f65e1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:22:05.441504 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:22:05.778763 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=68,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:22:18.824264 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:22:18.825698 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:22:18.838218 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a14c04-23f3-40b7-8170-452e311cb8d6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:22:18.897975 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:22:18.898598 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:22:18.898872 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:22:18.899168 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] PATH_INFO: `/v3/users/037f63576e404537b4e969487ab47231` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:22:18.900276 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=037f63576e404537b4e969487ab47231)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:22:18.902773 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:22:18.950473 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '037f63576e404537b4e969487ab47231'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:22:18.962540 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '037f63576e404537b4e969487ab47231'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:22:18.969899 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '037f63576e404537b4e969487ab47231'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:22:18.977182 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '037f63576e404537b4e969487ab47231'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:22:18.977798 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '037f63576e404537b4e969487ab47231'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:22:18.983704 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1484/2937] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 22 17:22:18 2022] DELETE /identity/v3/users/037f63576e404537b4e969487ab47231 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:22:18.988210 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ad544-4b90-4b55-b3d5-515341a57962 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:22:18.988718 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ad544-4b90-4b55-b3d5-515341a57962 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:22:18.988826 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ad544-4b90-4b55-b3d5-515341a57962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:22:19.007784 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c38ad544-4b90-4b55-b3d5-515341a57962 None None] MFA Rules not processed for user `d723a6acb5a24745b0576ef9d3892945`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 17:22:19.008969 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c38ad544-4b90-4b55-b3d5-515341a57962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:22:19.068164 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1454/2938] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 22 17:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 17:22:19.078724 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:22:19.080172 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:22:19.138892 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2922f26-4a5e-4f25-9a91-ebd3cb91b8e0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:22:19.142606 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00211911-41f9-4e92-be94-bf18327b2075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c2ed1eb664348e384824f39ca90c950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:22:19.143253 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00211911-41f9-4e92-be94-bf18327b2075 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:22:19.143535 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00211911-41f9-4e92-be94-bf18327b2075 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:22:19.143820 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00211911-41f9-4e92-be94-bf18327b2075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:22:19.154055 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00211911-41f9-4e92-be94-bf18327b2075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:22:19.156829 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00211911-41f9-4e92-be94-bf18327b2075 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:22:19.173082 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1485/2939] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:22:19.441312 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ad544-4b90-4b55-b3d5-515341a57962 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:22:19.442861 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c38ad544-4b90-4b55-b3d5-515341a57962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:22:19.455226 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ad544-4b90-4b55-b3d5-515341a57962 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:22:19.459881 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd723a6acb5a24745b0576ef9d3892945', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa999029c6348189112ae4315aae1bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:22:19.460506 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:22:19.460785 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:22:19.461056 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] PATH_INFO: `/v3/projects/84d17850960b4ef8b4655554158c3ff9` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:22:19.462196 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84d17850960b4ef8b4655554158c3ff9)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:22:19.464217 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:22:19.512165 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84d17850960b4ef8b4655554158c3ff9', 'request_id': 'req-e9fd0340-2338-4fc0-a247-5b4df31f5560'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:22:19.521128 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84d17850960b4ef8b4655554158c3ff9', 'request_id': 'req-e9fd0340-2338-4fc0-a247-5b4df31f5560'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:22:19.531334 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84d17850960b4ef8b4655554158c3ff9', 'request_id': 'req-e9fd0340-2338-4fc0-a247-5b4df31f5560'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:22:19.545631 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] The token cache is being invalidate because project 84d17850960b4ef8b4655554158c3ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 17:22:19.545913 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:22:19.546527 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e9fd0340-2338-4fc0-a247-5b4df31f5560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 17:22:19.548055 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1455/2940] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 22 17:22:19 2022] DELETE /identity/v3/projects/84d17850960b4ef8b4655554158c3ff9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 17:22:29.164101 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:22:29.541822 dv-cinder-835384-28-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 17:26:19.366890 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00211911-41f9-4e92-be94-bf18327b2075 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 17:26:19.368349 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-00211911-41f9-4e92-be94-bf18327b2075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:26:19.506210 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00211911-41f9-4e92-be94-bf18327b2075 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 17:26:19.568310 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9964f-9270-4b08-be49-dc39599e3356 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81088a327fd448ad8fec555f3d914be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2b73d7ceaff442fa3d7215afba1b5c1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 17:26:19.568964 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9964f-9270-4b08-be49-dc39599e3356 service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 17:26:19.569279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9964f-9270-4b08-be49-dc39599e3356 service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 17:26:19.569692 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9964f-9270-4b08-be49-dc39599e3356 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 17:26:19.571847 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3fd9964f-9270-4b08-be49-dc39599e3356 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 17:26:19.584358 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9964f-9270-4b08-be49-dc39599e3356 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 17:26:19.585952 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9964f-9270-4b08-be49-dc39599e3356 service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 17:26:19.597279 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1486/2941] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 22 17:26:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 17:26:29.592071 dv-cinder-835384-28-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}